2023-08-15 15:58:09,943 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42 2023-08-15 15:58:09,958 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling timeout: 13 mins 2023-08-15 15:58:09,992 INFO [Time-limited test] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=139, ProcessCount=169, AvailableMemoryMB=1686 2023-08-15 15:58:09,998 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-15 15:58:09,999 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f, deleteOnExit=true 2023-08-15 15:58:09,999 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-15 15:58:09,999 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/test.cache.data in system properties and HBase conf 2023-08-15 15:58:10,000 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/hadoop.tmp.dir in system properties and HBase conf 2023-08-15 15:58:10,000 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/hadoop.log.dir in system properties and HBase conf 2023-08-15 15:58:10,001 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-15 15:58:10,001 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-15 15:58:10,001 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-15 15:58:10,108 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-15 15:58:10,502 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-15 15:58:10,505 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-15 15:58:10,506 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-15 15:58:10,506 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-15 15:58:10,506 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-15 15:58:10,507 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-15 15:58:10,507 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-15 15:58:10,507 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-15 15:58:10,508 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-15 15:58:10,508 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-15 15:58:10,508 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/nfs.dump.dir in system properties and HBase conf 2023-08-15 15:58:10,508 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/java.io.tmpdir in system properties and HBase conf 2023-08-15 15:58:10,509 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-15 15:58:10,509 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-15 15:58:10,509 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-15 15:58:10,983 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-15 15:58:10,987 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-15 15:58:11,271 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-15 15:58:11,438 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-15 15:58:11,451 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:11,487 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:11,520 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/java.io.tmpdir/Jetty_localhost_34735_hdfs____.p629oe/webapp 2023-08-15 15:58:11,653 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34735 2023-08-15 15:58:11,671 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-15 15:58:11,672 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-15 15:58:12,155 WARN [Listener at localhost/39847] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:12,220 WARN [Listener at localhost/39847] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:58:12,237 WARN [Listener at localhost/39847] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:12,244 INFO [Listener at localhost/39847] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:12,248 INFO [Listener at localhost/39847] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/java.io.tmpdir/Jetty_localhost_36481_datanode____.vwvnzj/webapp 2023-08-15 15:58:12,344 INFO [Listener at localhost/39847] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36481 2023-08-15 15:58:12,651 WARN [Listener at localhost/36231] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:12,661 WARN [Listener at localhost/36231] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:58:12,665 WARN [Listener at localhost/36231] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:12,667 INFO [Listener at localhost/36231] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:12,673 INFO [Listener at localhost/36231] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/java.io.tmpdir/Jetty_localhost_37905_datanode____.aa1j3z/webapp 2023-08-15 15:58:12,773 INFO [Listener at localhost/36231] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37905 2023-08-15 15:58:12,783 WARN [Listener at localhost/42207] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:13,080 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb1e71832a6bd076f: Processing first storage report for DS-076a99ce-3a25-4acb-939c-2423e99621b1 from datanode ad129d24-8775-4701-8450-6f8107dfebeb 2023-08-15 15:58:13,081 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb1e71832a6bd076f: from storage DS-076a99ce-3a25-4acb-939c-2423e99621b1 node DatanodeRegistration(127.0.0.1:44549, datanodeUuid=ad129d24-8775-4701-8450-6f8107dfebeb, infoPort=37319, infoSecurePort=0, ipcPort=42207, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-15 15:58:13,081 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd87e2b285a7740cd: Processing first storage report for DS-24a2773a-0f47-49f3-9212-9f3e5070ec89 from datanode f9897934-fc8c-4e88-a4de-37bac7fcb431 2023-08-15 15:58:13,081 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd87e2b285a7740cd: from storage DS-24a2773a-0f47-49f3-9212-9f3e5070ec89 node DatanodeRegistration(127.0.0.1:41347, datanodeUuid=f9897934-fc8c-4e88-a4de-37bac7fcb431, infoPort=44987, infoSecurePort=0, ipcPort=36231, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:13,082 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb1e71832a6bd076f: Processing first storage report for DS-76046738-37ba-4095-9ade-b76b94f77fd5 from datanode ad129d24-8775-4701-8450-6f8107dfebeb 2023-08-15 15:58:13,082 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb1e71832a6bd076f: from storage DS-76046738-37ba-4095-9ade-b76b94f77fd5 node DatanodeRegistration(127.0.0.1:44549, datanodeUuid=ad129d24-8775-4701-8450-6f8107dfebeb, infoPort=37319, infoSecurePort=0, ipcPort=42207, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:13,082 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd87e2b285a7740cd: Processing first storage report for DS-1d02819e-74bd-4637-8d85-3aa83a18816e from datanode f9897934-fc8c-4e88-a4de-37bac7fcb431 2023-08-15 15:58:13,082 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd87e2b285a7740cd: from storage DS-1d02819e-74bd-4637-8d85-3aa83a18816e node DatanodeRegistration(127.0.0.1:41347, datanodeUuid=f9897934-fc8c-4e88-a4de-37bac7fcb431, infoPort=44987, infoSecurePort=0, ipcPort=36231, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:13,157 DEBUG [Listener at localhost/42207] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42 2023-08-15 15:58:13,222 INFO [Listener at localhost/42207] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/zookeeper_0, clientPort=53870, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-15 15:58:13,235 INFO [Listener at localhost/42207] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=53870 2023-08-15 15:58:13,248 INFO [Listener at localhost/42207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:13,250 INFO [Listener at localhost/42207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:13,928 INFO [Listener at localhost/42207] util.FSUtils(471): Created version file at hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528 with version=8 2023-08-15 15:58:13,928 INFO [Listener at localhost/42207] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/hbase-staging 2023-08-15 15:58:14,258 INFO [Listener at localhost/42207] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-15 15:58:14,745 INFO [Listener at localhost/42207] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-15 15:58:14,777 INFO [Listener at localhost/42207] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:14,778 INFO [Listener at localhost/42207] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:14,778 INFO [Listener at localhost/42207] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-15 15:58:14,778 INFO [Listener at localhost/42207] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:14,779 INFO [Listener at localhost/42207] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-15 15:58:14,920 INFO [Listener at localhost/42207] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-15 15:58:14,991 DEBUG [Listener at localhost/42207] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-15 15:58:15,084 INFO [Listener at localhost/42207] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38353 2023-08-15 15:58:15,094 INFO [Listener at localhost/42207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:15,097 INFO [Listener at localhost/42207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:15,118 INFO [Listener at localhost/42207] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38353 connecting to ZooKeeper ensemble=127.0.0.1:53870 2023-08-15 15:58:15,158 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:383530x0, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-15 15:58:15,160 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38353-0x10010aa7e7a0000 connected 2023-08-15 15:58:15,184 DEBUG [Listener at localhost/42207] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 15:58:15,185 DEBUG [Listener at localhost/42207] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:58:15,188 DEBUG [Listener at localhost/42207] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-15 15:58:15,196 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38353 2023-08-15 15:58:15,196 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38353 2023-08-15 15:58:15,196 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38353 2023-08-15 15:58:15,197 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38353 2023-08-15 15:58:15,197 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38353 2023-08-15 15:58:15,203 INFO [Listener at localhost/42207] master.HMaster(444): hbase.rootdir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528, hbase.cluster.distributed=false 2023-08-15 15:58:15,268 INFO [Listener at localhost/42207] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-15 15:58:15,268 INFO [Listener at localhost/42207] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:15,268 INFO [Listener at localhost/42207] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:15,269 INFO [Listener at localhost/42207] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-15 15:58:15,269 INFO [Listener at localhost/42207] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:15,269 INFO [Listener at localhost/42207] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-15 15:58:15,273 INFO [Listener at localhost/42207] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-15 15:58:15,276 INFO [Listener at localhost/42207] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36555 2023-08-15 15:58:15,278 INFO [Listener at localhost/42207] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-15 15:58:15,284 DEBUG [Listener at localhost/42207] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-15 15:58:15,285 INFO [Listener at localhost/42207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:15,287 INFO [Listener at localhost/42207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:15,289 INFO [Listener at localhost/42207] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36555 connecting to ZooKeeper ensemble=127.0.0.1:53870 2023-08-15 15:58:15,293 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): regionserver:365550x0, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-15 15:58:15,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36555-0x10010aa7e7a0001 connected 2023-08-15 15:58:15,294 DEBUG [Listener at localhost/42207] zookeeper.ZKUtil(164): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 15:58:15,295 DEBUG [Listener at localhost/42207] zookeeper.ZKUtil(164): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:58:15,296 DEBUG [Listener at localhost/42207] zookeeper.ZKUtil(164): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-15 15:58:15,297 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36555 2023-08-15 15:58:15,297 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36555 2023-08-15 15:58:15,297 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36555 2023-08-15 15:58:15,298 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36555 2023-08-15 15:58:15,298 DEBUG [Listener at localhost/42207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36555 2023-08-15 15:58:15,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:15,309 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-15 15:58:15,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:15,332 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-15 15:58:15,332 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-15 15:58:15,332 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:15,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-15 15:58:15,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38353,1692115094090 from backup master directory 2023-08-15 15:58:15,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-15 15:58:15,337 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:15,337 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-15 15:58:15,338 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-15 15:58:15,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:15,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-15 15:58:15,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-15 15:58:15,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/hbase.id with ID: 0e71fbb3-3860-4532-b288-cb0605a3533c 2023-08-15 15:58:15,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:15,482 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:15,524 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x29aee72d to 127.0.0.1:53870 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:58:15,557 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21f774e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:58:15,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-15 15:58:15,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-15 15:58:15,591 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:58:15,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store-tmp 2023-08-15 15:58:15,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:15,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-15 15:58:15,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:15,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:15,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-15 15:58:15,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:15,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:15,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 15:58:15,661 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:15,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38353%2C1692115094090, suffix=, logDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090, archiveDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/oldWALs, maxLogs=10 2023-08-15 15:58:15,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-15 15:58:15,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-15 15:58:15,714 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:15,747 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:15,749 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44549,DS-076a99ce-3a25-4acb-939c-2423e99621b1,DISK] 2023-08-15 15:58:15,749 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41347,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK] 2023-08-15 15:58:15,756 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:15,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 2023-08-15 15:58:15,818 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44549,DS-076a99ce-3a25-4acb-939c-2423e99621b1,DISK], DatanodeInfoWithStorage[127.0.0.1:41347,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK]] 2023-08-15 15:58:15,819 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:58:15,819 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:15,822 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:15,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:15,879 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:15,886 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-15 15:58:15,909 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-15 15:58:15,920 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:15,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:15,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:15,942 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:15,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:58:15,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=742718, jitterRate=-0.05558645725250244}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:58:15,947 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 15:58:15,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-15 15:58:15,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-15 15:58:15,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-15 15:58:15,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-15 15:58:15,971 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-15 15:58:16,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-08-15 15:58:16,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-15 15:58:16,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-15 15:58:16,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-15 15:58:16,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-15 15:58:16,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-15 15:58:16,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-15 15:58:16,067 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-15 15:58:16,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-15 15:58:16,075 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:16,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-15 15:58:16,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-15 15:58:16,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-15 15:58:16,093 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-15 15:58:16,093 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-15 15:58:16,093 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:16,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38353,1692115094090, sessionid=0x10010aa7e7a0000, setting cluster-up flag (Was=false) 2023-08-15 15:58:16,112 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:16,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-15 15:58:16,120 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:16,126 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:16,131 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-15 15:58:16,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:16,135 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.hbase-snapshot/.tmp 2023-08-15 15:58:16,202 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(951): ClusterId : 0e71fbb3-3860-4532-b288-cb0605a3533c 2023-08-15 15:58:16,207 DEBUG [RS:0;jenkins-hbase4:36555] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-15 15:58:16,214 DEBUG [RS:0;jenkins-hbase4:36555] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-15 15:58:16,214 DEBUG [RS:0;jenkins-hbase4:36555] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-15 15:58:16,218 DEBUG [RS:0;jenkins-hbase4:36555] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-15 15:58:16,219 DEBUG [RS:0;jenkins-hbase4:36555] zookeeper.ReadOnlyZKClient(139): Connect 0x0a8e602f to 127.0.0.1:53870 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:58:16,224 DEBUG [RS:0;jenkins-hbase4:36555] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4498f0e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:58:16,226 DEBUG [RS:0;jenkins-hbase4:36555] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@760b5f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-15 15:58:16,247 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-15 15:58:16,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:58:16,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:58:16,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:58:16,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:58:16,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-15 15:58:16,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-15 15:58:16,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692115126259 2023-08-15 15:58:16,260 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36555 2023-08-15 15:58:16,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-15 15:58:16,265 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-15 15:58:16,265 INFO [RS:0;jenkins-hbase4:36555] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-15 15:58:16,265 INFO [RS:0;jenkins-hbase4:36555] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-15 15:58:16,265 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-15 15:58:16,265 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1022): About to register with Master. 2023-08-15 15:58:16,268 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,38353,1692115094090 with isa=jenkins-hbase4.apache.org/172.31.14.131:36555, startcode=1692115095267 2023-08-15 15:58:16,270 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-15 15:58:16,271 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-15 15:58:16,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-15 15:58:16,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-15 15:58:16,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-15 15:58:16,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-15 15:58:16,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-15 15:58:16,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-15 15:58:16,284 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-15 15:58:16,286 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-15 15:58:16,286 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-15 15:58:16,289 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115096287,5,FailOnTimeoutGroup] 2023-08-15 15:58:16,289 DEBUG [RS:0;jenkins-hbase4:36555] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-15 15:58:16,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115096289,5,FailOnTimeoutGroup] 2023-08-15 15:58:16,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-15 15:58:16,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,313 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-15 15:58:16,314 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-15 15:58:16,314 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528 2023-08-15 15:58:16,336 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:16,339 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-15 15:58:16,343 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/info 2023-08-15 15:58:16,344 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-15 15:58:16,345 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:16,346 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-15 15:58:16,348 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:58:16,349 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-15 15:58:16,350 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:16,350 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-15 15:58:16,352 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/table 2023-08-15 15:58:16,353 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-15 15:58:16,354 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:16,354 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58015, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-15 15:58:16,355 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740 2023-08-15 15:58:16,356 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740 2023-08-15 15:58:16,360 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-15 15:58:16,362 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-15 15:58:16,367 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38353] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:16,368 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:58:16,369 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=833830, jitterRate=0.060270413756370544}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-15 15:58:16,369 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-15 15:58:16,369 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-15 15:58:16,369 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-15 15:58:16,369 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-15 15:58:16,369 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-15 15:58:16,370 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-15 15:58:16,371 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-15 15:58:16,371 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-15 15:58:16,378 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-15 15:58:16,379 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-15 15:58:16,387 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528 2023-08-15 15:58:16,387 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39847 2023-08-15 15:58:16,387 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-15 15:58:16,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-15 15:58:16,394 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 15:58:16,394 DEBUG [RS:0;jenkins-hbase4:36555] zookeeper.ZKUtil(162): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:16,395 WARN [RS:0;jenkins-hbase4:36555] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-15 15:58:16,395 INFO [RS:0;jenkins-hbase4:36555] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:58:16,395 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:16,396 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36555,1692115095267] 2023-08-15 15:58:16,404 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-15 15:58:16,405 DEBUG [RS:0;jenkins-hbase4:36555] zookeeper.ZKUtil(162): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:16,406 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-15 15:58:16,415 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-15 15:58:16,422 INFO [RS:0;jenkins-hbase4:36555] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-15 15:58:16,440 INFO [RS:0;jenkins-hbase4:36555] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-15 15:58:16,444 INFO [RS:0;jenkins-hbase4:36555] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-15 15:58:16,444 INFO [RS:0;jenkins-hbase4:36555] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,445 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-15 15:58:16,451 INFO [RS:0;jenkins-hbase4:36555] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,452 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,453 DEBUG [RS:0;jenkins-hbase4:36555] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:16,453 INFO [RS:0;jenkins-hbase4:36555] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,453 INFO [RS:0;jenkins-hbase4:36555] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,454 INFO [RS:0;jenkins-hbase4:36555] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,469 INFO [RS:0;jenkins-hbase4:36555] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-15 15:58:16,471 INFO [RS:0;jenkins-hbase4:36555] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36555,1692115095267-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:16,487 INFO [RS:0;jenkins-hbase4:36555] regionserver.Replication(203): jenkins-hbase4.apache.org,36555,1692115095267 started 2023-08-15 15:58:16,487 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36555,1692115095267, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36555, sessionid=0x10010aa7e7a0001 2023-08-15 15:58:16,487 DEBUG [RS:0;jenkins-hbase4:36555] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-15 15:58:16,487 DEBUG [RS:0;jenkins-hbase4:36555] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:16,487 DEBUG [RS:0;jenkins-hbase4:36555] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36555,1692115095267' 2023-08-15 15:58:16,488 DEBUG [RS:0;jenkins-hbase4:36555] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:58:16,488 DEBUG [RS:0;jenkins-hbase4:36555] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:58:16,489 DEBUG [RS:0;jenkins-hbase4:36555] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-15 15:58:16,489 DEBUG [RS:0;jenkins-hbase4:36555] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-15 15:58:16,489 DEBUG [RS:0;jenkins-hbase4:36555] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:16,489 DEBUG [RS:0;jenkins-hbase4:36555] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36555,1692115095267' 2023-08-15 15:58:16,489 DEBUG [RS:0;jenkins-hbase4:36555] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-15 15:58:16,489 DEBUG [RS:0;jenkins-hbase4:36555] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-15 15:58:16,490 DEBUG [RS:0;jenkins-hbase4:36555] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-15 15:58:16,490 INFO [RS:0;jenkins-hbase4:36555] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-15 15:58:16,490 INFO [RS:0;jenkins-hbase4:36555] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-15 15:58:16,558 DEBUG [jenkins-hbase4:38353] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-15 15:58:16,561 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36555,1692115095267, state=OPENING 2023-08-15 15:58:16,570 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-15 15:58:16,572 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:16,573 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-15 15:58:16,576 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36555,1692115095267}] 2023-08-15 15:58:16,600 INFO [RS:0;jenkins-hbase4:36555] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36555%2C1692115095267, suffix=, logDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267, archiveDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/oldWALs, maxLogs=32 2023-08-15 15:58:16,619 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41347,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK] 2023-08-15 15:58:16,619 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44549,DS-076a99ce-3a25-4acb-939c-2423e99621b1,DISK] 2023-08-15 15:58:16,623 INFO [RS:0;jenkins-hbase4:36555] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 2023-08-15 15:58:16,623 DEBUG [RS:0;jenkins-hbase4:36555] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41347,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK], DatanodeInfoWithStorage[127.0.0.1:44549,DS-076a99ce-3a25-4acb-939c-2423e99621b1,DISK]] 2023-08-15 15:58:16,760 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:16,763 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-15 15:58:16,766 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50102, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-15 15:58:16,779 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-15 15:58:16,780 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:58:16,784 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta, suffix=.meta, logDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267, archiveDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/oldWALs, maxLogs=32 2023-08-15 15:58:16,801 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44549,DS-076a99ce-3a25-4acb-939c-2423e99621b1,DISK] 2023-08-15 15:58:16,801 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41347,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK] 2023-08-15 15:58:16,806 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta 2023-08-15 15:58:16,806 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44549,DS-076a99ce-3a25-4acb-939c-2423e99621b1,DISK], DatanodeInfoWithStorage[127.0.0.1:41347,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK]] 2023-08-15 15:58:16,806 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:58:16,808 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-15 15:58:16,824 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-15 15:58:16,829 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-15 15:58:16,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-15 15:58:16,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:16,835 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-15 15:58:16,835 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-15 15:58:16,838 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-15 15:58:16,839 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/info 2023-08-15 15:58:16,840 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/info 2023-08-15 15:58:16,840 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-15 15:58:16,841 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:16,841 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-15 15:58:16,843 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:58:16,843 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:58:16,843 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-15 15:58:16,844 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:16,844 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-15 15:58:16,846 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/table 2023-08-15 15:58:16,846 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/table 2023-08-15 15:58:16,847 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-15 15:58:16,847 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:16,850 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740 2023-08-15 15:58:16,852 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740 2023-08-15 15:58:16,856 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-15 15:58:16,858 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-15 15:58:16,859 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=694938, jitterRate=-0.11634162068367004}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-15 15:58:16,859 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-15 15:58:16,869 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692115096753 2023-08-15 15:58:16,885 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-15 15:58:16,886 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-15 15:58:16,887 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36555,1692115095267, state=OPEN 2023-08-15 15:58:16,890 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-15 15:58:16,890 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-15 15:58:16,894 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-15 15:58:16,894 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36555,1692115095267 in 314 msec 2023-08-15 15:58:16,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-15 15:58:16,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 507 msec 2023-08-15 15:58:16,903 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 725 msec 2023-08-15 15:58:16,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692115096903, completionTime=-1 2023-08-15 15:58:16,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-15 15:58:16,904 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-15 15:58:16,966 DEBUG [hconnection-0x6a31d487-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-15 15:58:16,968 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50112, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-15 15:58:16,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-15 15:58:16,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692115156984 2023-08-15 15:58:16,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692115216984 2023-08-15 15:58:16,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 80 msec 2023-08-15 15:58:17,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38353,1692115094090-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:17,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38353,1692115094090-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:17,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38353,1692115094090-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:17,009 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38353, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:17,010 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:17,015 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-15 15:58:17,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-15 15:58:17,024 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-15 15:58:17,034 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-15 15:58:17,036 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-15 15:58:17,038 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-15 15:58:17,060 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,062 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868 empty. 2023-08-15 15:58:17,063 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,063 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-15 15:58:17,091 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-15 15:58:17,093 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 395cb94e01779d5dd9d13a78d2a94868, NAME => 'hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp 2023-08-15 15:58:17,108 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:17,108 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 395cb94e01779d5dd9d13a78d2a94868, disabling compactions & flushes 2023-08-15 15:58:17,108 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:17,108 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:17,108 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. after waiting 0 ms 2023-08-15 15:58:17,108 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:17,108 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:17,108 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 395cb94e01779d5dd9d13a78d2a94868: 2023-08-15 15:58:17,112 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-15 15:58:17,127 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115097115"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115097115"}]},"ts":"1692115097115"} 2023-08-15 15:58:17,153 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-15 15:58:17,155 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-15 15:58:17,159 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115097155"}]},"ts":"1692115097155"} 2023-08-15 15:58:17,162 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-15 15:58:17,175 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=395cb94e01779d5dd9d13a78d2a94868, ASSIGN}] 2023-08-15 15:58:17,177 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=395cb94e01779d5dd9d13a78d2a94868, ASSIGN 2023-08-15 15:58:17,179 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=395cb94e01779d5dd9d13a78d2a94868, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36555,1692115095267; forceNewPlan=false, retain=false 2023-08-15 15:58:17,330 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=395cb94e01779d5dd9d13a78d2a94868, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:17,330 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115097330"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115097330"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115097330"}]},"ts":"1692115097330"} 2023-08-15 15:58:17,335 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 395cb94e01779d5dd9d13a78d2a94868, server=jenkins-hbase4.apache.org,36555,1692115095267}] 2023-08-15 15:58:17,496 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:17,497 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 395cb94e01779d5dd9d13a78d2a94868, NAME => 'hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:58:17,499 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,500 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:17,500 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,500 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,502 INFO [StoreOpener-395cb94e01779d5dd9d13a78d2a94868-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,504 DEBUG [StoreOpener-395cb94e01779d5dd9d13a78d2a94868-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868/info 2023-08-15 15:58:17,504 DEBUG [StoreOpener-395cb94e01779d5dd9d13a78d2a94868-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868/info 2023-08-15 15:58:17,505 INFO [StoreOpener-395cb94e01779d5dd9d13a78d2a94868-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 395cb94e01779d5dd9d13a78d2a94868 columnFamilyName info 2023-08-15 15:58:17,506 INFO [StoreOpener-395cb94e01779d5dd9d13a78d2a94868-1] regionserver.HStore(310): Store=395cb94e01779d5dd9d13a78d2a94868/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:17,507 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,508 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,512 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:17,516 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:58:17,517 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 395cb94e01779d5dd9d13a78d2a94868; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=767222, jitterRate=-0.024427026510238647}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:58:17,517 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 395cb94e01779d5dd9d13a78d2a94868: 2023-08-15 15:58:17,519 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868., pid=6, masterSystemTime=1692115097488 2023-08-15 15:58:17,523 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:17,523 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:17,524 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=395cb94e01779d5dd9d13a78d2a94868, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:17,525 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115097524"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115097524"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115097524"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115097524"}]},"ts":"1692115097524"} 2023-08-15 15:58:17,532 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-15 15:58:17,532 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 395cb94e01779d5dd9d13a78d2a94868, server=jenkins-hbase4.apache.org,36555,1692115095267 in 193 msec 2023-08-15 15:58:17,536 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-15 15:58:17,536 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=395cb94e01779d5dd9d13a78d2a94868, ASSIGN in 357 msec 2023-08-15 15:58:17,537 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-15 15:58:17,538 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115097537"}]},"ts":"1692115097537"} 2023-08-15 15:58:17,540 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-15 15:58:17,545 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-15 15:58:17,548 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 520 msec 2023-08-15 15:58:17,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-15 15:58:17,638 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:58:17,638 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:17,678 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-15 15:58:17,696 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:58:17,702 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 34 msec 2023-08-15 15:58:17,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-15 15:58:17,726 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:58:17,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-08-15 15:58:17,739 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-15 15:58:17,742 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-15 15:58:17,742 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.404sec 2023-08-15 15:58:17,744 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-15 15:58:17,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-15 15:58:17,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-15 15:58:17,747 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38353,1692115094090-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-15 15:58:17,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38353,1692115094090-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-15 15:58:17,759 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-15 15:58:17,808 DEBUG [Listener at localhost/42207] zookeeper.ReadOnlyZKClient(139): Connect 0x1a951b2c to 127.0.0.1:53870 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:58:17,813 DEBUG [Listener at localhost/42207] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69580873, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:58:17,850 DEBUG [hconnection-0x35c90b1d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-15 15:58:17,861 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50126, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-15 15:58:17,871 INFO [Listener at localhost/42207] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:17,872 INFO [Listener at localhost/42207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:17,880 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-15 15:58:17,880 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:17,881 INFO [Listener at localhost/42207] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-15 15:58:17,894 WARN [Listener at localhost/42207] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:58:17,897 WARN [Listener at localhost/42207] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:17,898 INFO [Listener at localhost/42207] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:17,904 INFO [Listener at localhost/42207] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/java.io.tmpdir/Jetty_localhost_44217_datanode____.d3hdj5/webapp 2023-08-15 15:58:18,013 INFO [Listener at localhost/42207] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44217 2023-08-15 15:58:18,022 WARN [Listener at localhost/41885] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:18,044 WARN [Listener at localhost/41885] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:58:18,047 WARN [Listener at localhost/41885] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:18,048 INFO [Listener at localhost/41885] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:18,058 INFO [Listener at localhost/41885] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/java.io.tmpdir/Jetty_localhost_43195_datanode____.fqkl4b/webapp 2023-08-15 15:58:18,149 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbb8853b6dc8a5c5b: Processing first storage report for DS-72999fd1-bdeb-4f66-902c-e3b5268752ac from datanode 576c8855-0001-4bcf-9895-01f6b6a62741 2023-08-15 15:58:18,149 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbb8853b6dc8a5c5b: from storage DS-72999fd1-bdeb-4f66-902c-e3b5268752ac node DatanodeRegistration(127.0.0.1:32891, datanodeUuid=576c8855-0001-4bcf-9895-01f6b6a62741, infoPort=46025, infoSecurePort=0, ipcPort=41885, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-15 15:58:18,149 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbb8853b6dc8a5c5b: Processing first storage report for DS-93f66c82-0394-4a6f-acd1-4e5533289a96 from datanode 576c8855-0001-4bcf-9895-01f6b6a62741 2023-08-15 15:58:18,150 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbb8853b6dc8a5c5b: from storage DS-93f66c82-0394-4a6f-acd1-4e5533289a96 node DatanodeRegistration(127.0.0.1:32891, datanodeUuid=576c8855-0001-4bcf-9895-01f6b6a62741, infoPort=46025, infoSecurePort=0, ipcPort=41885, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:18,176 INFO [Listener at localhost/41885] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43195 2023-08-15 15:58:18,185 WARN [Listener at localhost/36669] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:18,204 WARN [Listener at localhost/36669] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:58:18,208 WARN [Listener at localhost/36669] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:18,209 INFO [Listener at localhost/36669] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:18,214 INFO [Listener at localhost/36669] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/java.io.tmpdir/Jetty_localhost_38027_datanode____el2itz/webapp 2023-08-15 15:58:18,303 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd367e63d2e2ad27f: Processing first storage report for DS-c5fea4e5-93a4-41e3-9479-17dac7c20913 from datanode a0ca08ea-4769-4661-b232-cdeaa2ab97d0 2023-08-15 15:58:18,303 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd367e63d2e2ad27f: from storage DS-c5fea4e5-93a4-41e3-9479-17dac7c20913 node DatanodeRegistration(127.0.0.1:41029, datanodeUuid=a0ca08ea-4769-4661-b232-cdeaa2ab97d0, infoPort=37423, infoSecurePort=0, ipcPort=36669, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:18,303 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd367e63d2e2ad27f: Processing first storage report for DS-beaad869-1766-406c-ab08-5f5168ad7f35 from datanode a0ca08ea-4769-4661-b232-cdeaa2ab97d0 2023-08-15 15:58:18,303 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd367e63d2e2ad27f: from storage DS-beaad869-1766-406c-ab08-5f5168ad7f35 node DatanodeRegistration(127.0.0.1:41029, datanodeUuid=a0ca08ea-4769-4661-b232-cdeaa2ab97d0, infoPort=37423, infoSecurePort=0, ipcPort=36669, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:18,326 INFO [Listener at localhost/36669] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38027 2023-08-15 15:58:18,335 WARN [Listener at localhost/37745] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:18,439 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x29d6e3668163e508: Processing first storage report for DS-f5c7ff89-aa92-45ed-ac48-0feb6a04bc17 from datanode e3445a32-77f5-47fb-89ed-128665acb9b7 2023-08-15 15:58:18,439 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x29d6e3668163e508: from storage DS-f5c7ff89-aa92-45ed-ac48-0feb6a04bc17 node DatanodeRegistration(127.0.0.1:36463, datanodeUuid=e3445a32-77f5-47fb-89ed-128665acb9b7, infoPort=37589, infoSecurePort=0, ipcPort=37745, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-15 15:58:18,439 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x29d6e3668163e508: Processing first storage report for DS-93d07800-630d-463b-b3b7-ebd36c623ed3 from datanode e3445a32-77f5-47fb-89ed-128665acb9b7 2023-08-15 15:58:18,439 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x29d6e3668163e508: from storage DS-93d07800-630d-463b-b3b7-ebd36c623ed3 node DatanodeRegistration(127.0.0.1:36463, datanodeUuid=e3445a32-77f5-47fb-89ed-128665acb9b7, infoPort=37589, infoSecurePort=0, ipcPort=37745, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:18,456 DEBUG [Listener at localhost/37745] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-15 15:58:18,460 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41294, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-15 15:58:18,468 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-15 15:58:18,468 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-15 15:58:18,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-15 15:58:18,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath 2023-08-15 15:58:18,476 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_PRE_OPERATION 2023-08-15 15:58:18,478 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-15 15:58:18,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRollOnDatanodeDeath" procId is: 9 2023-08-15 15:58:18,481 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,482 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3 empty. 2023-08-15 15:58:18,484 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,484 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRollOnDatanodeDeath regions 2023-08-15 15:58:18,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-15 15:58:18,507 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/.tabledesc/.tableinfo.0000000001 2023-08-15 15:58:18,509 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(7675): creating {ENCODED => 27a98593350bb7a1c06e40a6ff6643c3, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/.tmp 2023-08-15 15:58:18,526 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:18,527 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1604): Closing 27a98593350bb7a1c06e40a6ff6643c3, disabling compactions & flushes 2023-08-15 15:58:18,527 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:18,527 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:18,527 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. after waiting 0 ms 2023-08-15 15:58:18,527 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:18,527 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:18,527 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1558): Region close journal for 27a98593350bb7a1c06e40a6ff6643c3: 2023-08-15 15:58:18,531 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ADD_TO_META 2023-08-15 15:58:18,532 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692115098532"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115098532"}]},"ts":"1692115098532"} 2023-08-15 15:58:18,535 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-15 15:58:18,536 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-15 15:58:18,536 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115098536"}]},"ts":"1692115098536"} 2023-08-15 15:58:18,538 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLING in hbase:meta 2023-08-15 15:58:18,542 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=27a98593350bb7a1c06e40a6ff6643c3, ASSIGN}] 2023-08-15 15:58:18,544 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=27a98593350bb7a1c06e40a6ff6643c3, ASSIGN 2023-08-15 15:58:18,546 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=27a98593350bb7a1c06e40a6ff6643c3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36555,1692115095267; forceNewPlan=false, retain=false 2023-08-15 15:58:18,698 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=27a98593350bb7a1c06e40a6ff6643c3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:18,698 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692115098698"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115098698"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115098698"}]},"ts":"1692115098698"} 2023-08-15 15:58:18,701 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 27a98593350bb7a1c06e40a6ff6643c3, server=jenkins-hbase4.apache.org,36555,1692115095267}] 2023-08-15 15:58:18,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:18,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 27a98593350bb7a1c06e40a6ff6643c3, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:58:18,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRollOnDatanodeDeath 27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:18,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,865 INFO [StoreOpener-27a98593350bb7a1c06e40a6ff6643c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,867 DEBUG [StoreOpener-27a98593350bb7a1c06e40a6ff6643c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3/info 2023-08-15 15:58:18,867 DEBUG [StoreOpener-27a98593350bb7a1c06e40a6ff6643c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3/info 2023-08-15 15:58:18,867 INFO [StoreOpener-27a98593350bb7a1c06e40a6ff6643c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 27a98593350bb7a1c06e40a6ff6643c3 columnFamilyName info 2023-08-15 15:58:18,868 INFO [StoreOpener-27a98593350bb7a1c06e40a6ff6643c3-1] regionserver.HStore(310): Store=27a98593350bb7a1c06e40a6ff6643c3/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:18,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:18,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:58:18,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 27a98593350bb7a1c06e40a6ff6643c3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=720584, jitterRate=-0.08373121917247772}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:58:18,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 27a98593350bb7a1c06e40a6ff6643c3: 2023-08-15 15:58:18,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3., pid=11, masterSystemTime=1692115098855 2023-08-15 15:58:18,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:18,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:18,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=27a98593350bb7a1c06e40a6ff6643c3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:18,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692115098884"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115098884"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115098884"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115098884"}]},"ts":"1692115098884"} 2023-08-15 15:58:18,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-15 15:58:18,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 27a98593350bb7a1c06e40a6ff6643c3, server=jenkins-hbase4.apache.org,36555,1692115095267 in 186 msec 2023-08-15 15:58:18,893 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-15 15:58:18,894 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=27a98593350bb7a1c06e40a6ff6643c3, ASSIGN in 348 msec 2023-08-15 15:58:18,895 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-15 15:58:18,895 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115098895"}]},"ts":"1692115098895"} 2023-08-15 15:58:18,898 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLED in hbase:meta 2023-08-15 15:58:18,901 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_POST_OPERATION 2023-08-15 15:58:18,904 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath in 430 msec 2023-08-15 15:58:22,347 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-15 15:58:22,460 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-15 15:58:22,461 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-15 15:58:22,462 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRollOnDatanodeDeath' 2023-08-15 15:58:24,254 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-15 15:58:24,254 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-15 15:58:28,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-15 15:58:28,497 INFO [Listener at localhost/37745] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRollOnDatanodeDeath, procId: 9 completed 2023-08-15 15:58:28,498 DEBUG [Listener at localhost/37745] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestLogRolling-testLogRollOnDatanodeDeath get assigned. Timeout = 60000ms 2023-08-15 15:58:28,499 INFO [Listener at localhost/37745] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-08-15 15:58:28,503 INFO [Listener at localhost/37745] hbase.HBaseTestingUtility(3484): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned to meta. Checking AM states. 2023-08-15 15:58:28,504 INFO [Listener at localhost/37745] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-08-15 15:58:28,504 INFO [Listener at localhost/37745] hbase.HBaseTestingUtility(3504): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned. 2023-08-15 15:58:28,514 DEBUG [Listener at localhost/37745] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRollOnDatanodeDeath 2023-08-15 15:58:28,515 DEBUG [Listener at localhost/37745] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:28,515 WARN [Listener at localhost/37745] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 15:58:28,521 INFO [Listener at localhost/37745] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:58:28,524 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1457568952_17 at /127.0.0.1:53342 [Receiving block BP-1360484752-172.31.14.131-1692115091062:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:44549:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:53342 dst: /127.0.0.1:44549 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,524 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1457568952_17 at /127.0.0.1:53346 [Receiving block BP-1360484752-172.31.14.131-1692115091062:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:44549:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:53346 dst: /127.0.0.1:44549 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,524 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-347186212_17 at /127.0.0.1:53310 [Receiving block BP-1360484752-172.31.14.131-1692115091062:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:44549:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:53310 dst: /127.0.0.1:44549 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,623 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1457568952_17 at /127.0.0.1:41774 [Receiving block BP-1360484752-172.31.14.131-1692115091062:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:41347:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:41774 dst: /127.0.0.1:41347 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,625 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1457568952_17 at /127.0.0.1:41764 [Receiving block BP-1360484752-172.31.14.131-1692115091062:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:41347:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:41764 dst: /127.0.0.1:41347 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,624 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-347186212_17 at /127.0.0.1:41746 [Receiving block BP-1360484752-172.31.14.131-1692115091062:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:41347:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:41746 dst: /127.0.0.1:41347 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,626 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 15:58:28,627 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1360484752-172.31.14.131-1692115091062 (Datanode Uuid f9897934-fc8c-4e88-a4de-37bac7fcb431) service to localhost/127.0.0.1:39847 2023-08-15 15:58:28,629 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data1/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:28,629 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data2/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:28,636 WARN [Listener at localhost/37745] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:58:28,638 WARN [Listener at localhost/37745] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:28,640 INFO [Listener at localhost/37745] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:28,644 INFO [Listener at localhost/37745] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/java.io.tmpdir/Jetty_localhost_39919_datanode____g3f142/webapp 2023-08-15 15:58:28,736 INFO [Listener at localhost/37745] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39919 2023-08-15 15:58:28,745 WARN [Listener at localhost/41683] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:28,749 WARN [AsyncFSWAL-0-hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528-prefix:jenkins-hbase4.apache.org,36555,1692115095267] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,750 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C36555%2C1692115095267:(num 1692115096602) roll requested 2023-08-15 15:58:28,767 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41029,DS-c5fea4e5-93a4-41e3-9479-17dac7c20913,DISK] 2023-08-15 15:58:28,768 WARN [regionserver/jenkins-hbase4:0.logRoller] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(571): create fan-out dfs output /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115108750 failed, retry = 0 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: /127.0.0.1:41347 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,844 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x259323e40ab5e096: Processing first storage report for DS-24a2773a-0f47-49f3-9212-9f3e5070ec89 from datanode f9897934-fc8c-4e88-a4de-37bac7fcb431 2023-08-15 15:58:28,845 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x259323e40ab5e096: from storage DS-24a2773a-0f47-49f3-9212-9f3e5070ec89 node DatanodeRegistration(127.0.0.1:32897, datanodeUuid=f9897934-fc8c-4e88-a4de-37bac7fcb431, infoPort=46177, infoSecurePort=0, ipcPort=41683, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 5, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-15 15:58:28,845 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x259323e40ab5e096: Processing first storage report for DS-1d02819e-74bd-4637-8d85-3aa83a18816e from datanode f9897934-fc8c-4e88-a4de-37bac7fcb431 2023-08-15 15:58:28,845 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x259323e40ab5e096: from storage DS-1d02819e-74bd-4637-8d85-3aa83a18816e node DatanodeRegistration(127.0.0.1:32897, datanodeUuid=f9897934-fc8c-4e88-a4de-37bac7fcb431, infoPort=46177, infoSecurePort=0, ipcPort=41683, storageInfo=lv=-57;cid=testClusterID;nsid=1327175493;c=1692115091062), blocks: 6, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-15 15:58:28,870 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_1457568952_17 at /127.0.0.1:33752 [Receiving block BP-1360484752-172.31.14.131-1692115091062:blk_1073741838_1014]] datanode.DataXceiver(323): 127.0.0.1:41029:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:33752 dst: /127.0.0.1:41029 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,876 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32897,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK] 2023-08-15 15:58:28,876 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41029,DS-c5fea4e5-93a4-41e3-9479-17dac7c20913,DISK] 2023-08-15 15:58:28,881 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 with entries=6, filesize=3.19 KB; new WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115108750 2023-08-15 15:58:28,881 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41029,DS-c5fea4e5-93a4-41e3-9479-17dac7c20913,DISK], DatanodeInfoWithStorage[127.0.0.1:32897,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK]] 2023-08-15 15:58:28,881 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 is not closed yet, will try archiving it next time 2023-08-15 15:58:28,883 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-15 15:58:28,883 INFO [Listener at localhost/41683] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-15 15:58:28,884 DEBUG [Listener at localhost/41683] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1a951b2c to 127.0.0.1:53870 2023-08-15 15:58:28,884 DEBUG [Listener at localhost/41683] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:58:28,885 DEBUG [Listener at localhost/41683] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-15 15:58:28,885 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,885 DEBUG [Listener at localhost/41683] util.JVMClusterUtil(257): Found active master hash=1649127331, stopped=false 2023-08-15 15:58:28,885 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,885 INFO [Listener at localhost/41683] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:28,888 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 2023-08-15 15:58:28,888 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-15 15:58:28,888 INFO [Listener at localhost/41683] procedure2.ProcedureExecutor(629): Stopping 2023-08-15 15:58:28,888 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:28,888 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-15 15:58:28,889 DEBUG [Listener at localhost/41683] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29aee72d to 127.0.0.1:53870 2023-08-15 15:58:28,889 DEBUG [Listener at localhost/41683] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:58:28,890 INFO [Listener at localhost/41683] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36555,1692115095267' ***** 2023-08-15 15:58:28,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:58:28,890 INFO [Listener at localhost/41683] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-15 15:58:28,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:58:28,890 INFO [RS:0;jenkins-hbase4:36555] regionserver.HeapMemoryManager(220): Stopping 2023-08-15 15:58:28,891 INFO [RS:0;jenkins-hbase4:36555] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-15 15:58:28,891 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-15 15:58:28,891 INFO [RS:0;jenkins-hbase4:36555] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-15 15:58:28,892 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(3305): Received CLOSE for 395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:28,892 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(3305): Received CLOSE for 27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:28,893 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:28,893 DEBUG [RS:0;jenkins-hbase4:36555] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0a8e602f to 127.0.0.1:53870 2023-08-15 15:58:28,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 395cb94e01779d5dd9d13a78d2a94868, disabling compactions & flushes 2023-08-15 15:58:28,893 DEBUG [RS:0;jenkins-hbase4:36555] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:58:28,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:28,894 INFO [RS:0;jenkins-hbase4:36555] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-15 15:58:28,894 INFO [RS:0;jenkins-hbase4:36555] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-15 15:58:28,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:28,894 INFO [RS:0;jenkins-hbase4:36555] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-15 15:58:28,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. after waiting 0 ms 2023-08-15 15:58:28,894 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-15 15:58:28,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:28,894 WARN [IPC Server handler 2 on default port 39847] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 has not been closed. Lease recovery is in progress. RecoveryId = 1016 for block blk_1073741832_1008 2023-08-15 15:58:28,894 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-15 15:58:28,895 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1478): Online Regions={395cb94e01779d5dd9d13a78d2a94868=hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868., 1588230740=hbase:meta,,1.1588230740, 27a98593350bb7a1c06e40a6ff6643c3=TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3.} 2023-08-15 15:58:28,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-15 15:58:28,895 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-15 15:58:28,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 395cb94e01779d5dd9d13a78d2a94868 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-15 15:58:28,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-15 15:58:28,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-15 15:58:28,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-15 15:58:28,896 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.92 KB heapSize=5.45 KB 2023-08-15 15:58:28,896 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1504): Waiting on 1588230740, 27a98593350bb7a1c06e40a6ff6643c3, 395cb94e01779d5dd9d13a78d2a94868 2023-08-15 15:58:28,897 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 after 9ms 2023-08-15 15:58:28,905 WARN [AsyncFSWAL-0-hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528-prefix:jenkins-hbase4.apache.org,36555,1692115095267.meta] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,906 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta:.meta(num 1692115096785) roll requested 2023-08-15 15:58:28,923 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44549,DS-076a99ce-3a25-4acb-939c-2423e99621b1,DISK] 2023-08-15 15:58:28,927 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32891,DS-72999fd1-bdeb-4f66-902c-e3b5268752ac,DISK] 2023-08-15 15:58:28,930 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta with entries=12, filesize=3.87 KB; new WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115108906.meta 2023-08-15 15:58:28,930 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44549,DS-076a99ce-3a25-4acb-939c-2423e99621b1,DISK], DatanodeInfoWithStorage[127.0.0.1:32891,DS-72999fd1-bdeb-4f66-902c-e3b5268752ac,DISK]] 2023-08-15 15:58:28,930 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta is not closed yet, will try archiving it next time 2023-08-15 15:58:28,930 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,931 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:28,932 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta 2023-08-15 15:58:28,933 WARN [IPC Server handler 0 on default port 39847] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta has not been closed. Lease recovery is in progress. RecoveryId = 1018 for block blk_1073741833_1009 2023-08-15 15:58:28,933 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta after 1ms 2023-08-15 15:58:28,976 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.69 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/.tmp/info/da399bc5ce38483d86344086fa75be48 2023-08-15 15:58:28,977 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868/.tmp/info/195c2cd07ae440a29f9383703ed96b18 2023-08-15 15:58:29,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868/.tmp/info/195c2cd07ae440a29f9383703ed96b18 as hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868/info/195c2cd07ae440a29f9383703ed96b18 2023-08-15 15:58:29,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868/info/195c2cd07ae440a29f9383703ed96b18, entries=2, sequenceid=6, filesize=4.8 K 2023-08-15 15:58:29,043 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 395cb94e01779d5dd9d13a78d2a94868 in 148ms, sequenceid=6, compaction requested=false 2023-08-15 15:58:29,047 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/.tmp/table/0151bc27540b4fb195b4c3882a030d70 2023-08-15 15:58:29,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/namespace/395cb94e01779d5dd9d13a78d2a94868/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-15 15:58:29,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:29,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 395cb94e01779d5dd9d13a78d2a94868: 2023-08-15 15:58:29,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692115097024.395cb94e01779d5dd9d13a78d2a94868. 2023-08-15 15:58:29,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 27a98593350bb7a1c06e40a6ff6643c3, disabling compactions & flushes 2023-08-15 15:58:29,056 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:29,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:29,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. after waiting 0 ms 2023-08-15 15:58:29,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:29,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 27a98593350bb7a1c06e40a6ff6643c3 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-08-15 15:58:29,059 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/.tmp/info/da399bc5ce38483d86344086fa75be48 as hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/info/da399bc5ce38483d86344086fa75be48 2023-08-15 15:58:29,070 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/info/da399bc5ce38483d86344086fa75be48, entries=20, sequenceid=14, filesize=7.4 K 2023-08-15 15:58:29,079 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/.tmp/table/0151bc27540b4fb195b4c3882a030d70 as hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/table/0151bc27540b4fb195b4c3882a030d70 2023-08-15 15:58:29,082 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3/.tmp/info/8090062accbd4cf2aec53d0afa2b8959 2023-08-15 15:58:29,090 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/table/0151bc27540b4fb195b4c3882a030d70, entries=4, sequenceid=14, filesize=4.8 K 2023-08-15 15:58:29,091 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.92 KB/2994, heapSize ~5.16 KB/5288, currentSize=0 B/0 for 1588230740 in 196ms, sequenceid=14, compaction requested=false 2023-08-15 15:58:29,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3/.tmp/info/8090062accbd4cf2aec53d0afa2b8959 as hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3/info/8090062accbd4cf2aec53d0afa2b8959 2023-08-15 15:58:29,096 DEBUG [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1504): Waiting on 1588230740, 27a98593350bb7a1c06e40a6ff6643c3 2023-08-15 15:58:29,101 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-08-15 15:58:29,102 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-15 15:58:29,103 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-15 15:58:29,103 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-15 15:58:29,103 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-15 15:58:29,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3/info/8090062accbd4cf2aec53d0afa2b8959, entries=2, sequenceid=6, filesize=6.8 K 2023-08-15 15:58:29,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for 27a98593350bb7a1c06e40a6ff6643c3 in 50ms, sequenceid=6, compaction requested=false 2023-08-15 15:58:29,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/data/default/TestLogRolling-testLogRollOnDatanodeDeath/27a98593350bb7a1c06e40a6ff6643c3/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-15 15:58:29,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:29,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 27a98593350bb7a1c06e40a6ff6643c3: 2023-08-15 15:58:29,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1692115098468.27a98593350bb7a1c06e40a6ff6643c3. 2023-08-15 15:58:29,297 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36555,1692115095267; all regions closed. 2023-08-15 15:58:29,507 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-15 15:58:29,507 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-15 15:58:30,455 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-15 15:58:32,898 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 after 4010ms 2023-08-15 15:58:32,899 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/oldWALs/jenkins-hbase4.apache.org%2C36555%2C1692115095267.1692115096602 2023-08-15 15:58:32,934 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta after 4002ms 2023-08-15 15:58:32,935 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/WALs/jenkins-hbase4.apache.org,36555,1692115095267/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/oldWALs/jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta.1692115096785.meta 2023-08-15 15:58:32,940 DEBUG [RS:0;jenkins-hbase4:36555] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/oldWALs 2023-08-15 15:58:32,940 INFO [RS:0;jenkins-hbase4:36555] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36555%2C1692115095267.meta:.meta(num 1692115108906) 2023-08-15 15:58:32,949 DEBUG [RS:0;jenkins-hbase4:36555] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/oldWALs 2023-08-15 15:58:32,949 INFO [RS:0;jenkins-hbase4:36555] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36555%2C1692115095267:(num 1692115108750) 2023-08-15 15:58:32,949 DEBUG [RS:0;jenkins-hbase4:36555] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:58:32,949 INFO [RS:0;jenkins-hbase4:36555] regionserver.LeaseManager(133): Closed leases 2023-08-15 15:58:32,950 INFO [RS:0;jenkins-hbase4:36555] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-15 15:58:32,950 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-15 15:58:32,951 INFO [RS:0;jenkins-hbase4:36555] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36555 2023-08-15 15:58:32,959 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36555,1692115095267 2023-08-15 15:58:32,959 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 15:58:32,959 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 15:58:32,961 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36555,1692115095267] 2023-08-15 15:58:32,962 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36555,1692115095267; numProcessing=1 2023-08-15 15:58:32,964 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36555,1692115095267 already deleted, retry=false 2023-08-15 15:58:32,964 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36555,1692115095267 expired; onlineServers=0 2023-08-15 15:58:32,964 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38353,1692115094090' ***** 2023-08-15 15:58:32,964 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-15 15:58:32,965 DEBUG [M:0;jenkins-hbase4:38353] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2376a598, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-15 15:58:32,965 INFO [M:0;jenkins-hbase4:38353] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38353,1692115094090 2023-08-15 15:58:32,965 INFO [M:0;jenkins-hbase4:38353] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38353,1692115094090; all regions closed. 2023-08-15 15:58:32,965 DEBUG [M:0;jenkins-hbase4:38353] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:58:32,965 DEBUG [M:0;jenkins-hbase4:38353] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-15 15:58:32,965 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-15 15:58:32,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115096289] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115096289,5,FailOnTimeoutGroup] 2023-08-15 15:58:32,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115096287] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115096287,5,FailOnTimeoutGroup] 2023-08-15 15:58:32,965 DEBUG [M:0;jenkins-hbase4:38353] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-15 15:58:32,967 INFO [M:0;jenkins-hbase4:38353] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-15 15:58:32,967 INFO [M:0;jenkins-hbase4:38353] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-15 15:58:32,967 INFO [M:0;jenkins-hbase4:38353] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-15 15:58:32,968 DEBUG [M:0;jenkins-hbase4:38353] master.HMaster(1512): Stopping service threads 2023-08-15 15:58:32,968 INFO [M:0;jenkins-hbase4:38353] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-15 15:58:32,968 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-15 15:58:32,969 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:32,969 ERROR [M:0;jenkins-hbase4:38353] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-08-15 15:58:32,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 15:58:32,969 INFO [M:0;jenkins-hbase4:38353] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-15 15:58:32,969 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-15 15:58:32,970 DEBUG [M:0;jenkins-hbase4:38353] zookeeper.ZKUtil(398): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-15 15:58:32,970 WARN [M:0;jenkins-hbase4:38353] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-15 15:58:32,970 INFO [M:0;jenkins-hbase4:38353] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-15 15:58:32,970 INFO [M:0;jenkins-hbase4:38353] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-15 15:58:32,971 DEBUG [M:0;jenkins-hbase4:38353] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-15 15:58:32,971 INFO [M:0;jenkins-hbase4:38353] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:32,971 DEBUG [M:0;jenkins-hbase4:38353] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:32,971 DEBUG [M:0;jenkins-hbase4:38353] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-15 15:58:32,971 DEBUG [M:0;jenkins-hbase4:38353] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:32,971 INFO [M:0;jenkins-hbase4:38353] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.09 KB heapSize=45.73 KB 2023-08-15 15:58:32,972 WARN [AsyncFSWAL-0-hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData-prefix:jenkins-hbase4.apache.org,38353,1692115094090] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:32,972 DEBUG [master:store-WAL-Roller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C38353%2C1692115094090:(num 1692115095691) roll requested 2023-08-15 15:58:32,992 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32897,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK] 2023-08-15 15:58:32,992 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32891,DS-72999fd1-bdeb-4f66-902c-e3b5268752ac,DISK] 2023-08-15 15:58:32,996 INFO [master:store-WAL-Roller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 with entries=89, filesize=43.94 KB; new WAL /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115112972 2023-08-15 15:58:32,996 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32891,DS-72999fd1-bdeb-4f66-902c-e3b5268752ac,DISK], DatanodeInfoWithStorage[127.0.0.1:32897,DS-24a2773a-0f47-49f3-9212-9f3e5070ec89,DISK]] 2023-08-15 15:58:32,996 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(716): hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 is not closed yet, will try archiving it next time 2023-08-15 15:58:32,996 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:32,996 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-08-15 15:58:32,996 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 2023-08-15 15:58:32,997 WARN [IPC Server handler 2 on default port 39847] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 has not been closed. Lease recovery is in progress. RecoveryId = 1024 for block blk_1073741829_1005 2023-08-15 15:58:32,998 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 after 2ms 2023-08-15 15:58:33,014 INFO [M:0;jenkins-hbase4:38353] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.09 KB at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5d6a9dea0a4346989a65d3c716301897 2023-08-15 15:58:33,023 DEBUG [M:0;jenkins-hbase4:38353] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5d6a9dea0a4346989a65d3c716301897 as hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5d6a9dea0a4346989a65d3c716301897 2023-08-15 15:58:33,032 INFO [M:0;jenkins-hbase4:38353] regionserver.HStore(1080): Added hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5d6a9dea0a4346989a65d3c716301897, entries=11, sequenceid=91, filesize=7.0 K 2023-08-15 15:58:33,033 INFO [M:0;jenkins-hbase4:38353] regionserver.HRegion(2948): Finished flush of dataSize ~38.09 KB/39009, heapSize ~45.72 KB/46816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 62ms, sequenceid=91, compaction requested=false 2023-08-15 15:58:33,034 INFO [M:0;jenkins-hbase4:38353] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:33,034 DEBUG [M:0;jenkins-hbase4:38353] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 15:58:33,062 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 15:58:33,062 INFO [RS:0;jenkins-hbase4:36555] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36555,1692115095267; zookeeper connection closed. 2023-08-15 15:58:33,062 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): regionserver:36555-0x10010aa7e7a0001, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 15:58:33,062 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1b350604] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1b350604 2023-08-15 15:58:33,063 INFO [Listener at localhost/41683] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-15 15:58:34,589 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-15 15:58:34,624 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-15 15:58:36,999 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 after 4003ms 2023-08-15 15:58:37,000 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/WALs/jenkins-hbase4.apache.org,38353,1692115094090/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/oldWALs/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 2023-08-15 15:58:37,006 INFO [WAL-Archive-0] region.MasterRegionUtils(50): Moved hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/MasterData/oldWALs/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691 to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/oldWALs/jenkins-hbase4.apache.org%2C38353%2C1692115094090.1692115095691$masterlocalwal$ 2023-08-15 15:58:37,006 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-15 15:58:37,006 INFO [M:0;jenkins-hbase4:38353] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-15 15:58:37,007 INFO [M:0;jenkins-hbase4:38353] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38353 2023-08-15 15:58:37,010 DEBUG [M:0;jenkins-hbase4:38353] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38353,1692115094090 already deleted, retry=false 2023-08-15 15:58:37,112 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 15:58:37,112 INFO [M:0;jenkins-hbase4:38353] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38353,1692115094090; zookeeper connection closed. 2023-08-15 15:58:37,112 DEBUG [Listener at localhost/42207-EventThread] zookeeper.ZKWatcher(600): master:38353-0x10010aa7e7a0000, quorum=127.0.0.1:53870, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 15:58:37,114 WARN [Listener at localhost/41683] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 15:58:37,117 INFO [Listener at localhost/41683] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:58:37,220 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 15:58:37,220 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1360484752-172.31.14.131-1692115091062 (Datanode Uuid f9897934-fc8c-4e88-a4de-37bac7fcb431) service to localhost/127.0.0.1:39847 2023-08-15 15:58:37,221 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data1/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,221 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data2/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,223 WARN [Listener at localhost/41683] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 15:58:37,228 INFO [Listener at localhost/41683] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:58:37,331 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 15:58:37,331 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1360484752-172.31.14.131-1692115091062 (Datanode Uuid e3445a32-77f5-47fb-89ed-128665acb9b7) service to localhost/127.0.0.1:39847 2023-08-15 15:58:37,332 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data9/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,332 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data10/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,334 WARN [Listener at localhost/41683] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 15:58:37,337 INFO [Listener at localhost/41683] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:58:37,440 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 15:58:37,440 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1360484752-172.31.14.131-1692115091062 (Datanode Uuid a0ca08ea-4769-4661-b232-cdeaa2ab97d0) service to localhost/127.0.0.1:39847 2023-08-15 15:58:37,441 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data7/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,441 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data8/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,443 WARN [Listener at localhost/41683] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 15:58:37,446 INFO [Listener at localhost/41683] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:58:37,549 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 15:58:37,549 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1360484752-172.31.14.131-1692115091062 (Datanode Uuid 576c8855-0001-4bcf-9895-01f6b6a62741) service to localhost/127.0.0.1:39847 2023-08-15 15:58:37,550 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data5/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,550 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data6/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,551 WARN [Listener at localhost/41683] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 15:58:37,553 INFO [Listener at localhost/41683] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:58:37,654 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 15:58:37,654 WARN [BP-1360484752-172.31.14.131-1692115091062 heartbeating to localhost/127.0.0.1:39847] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1360484752-172.31.14.131-1692115091062 (Datanode Uuid ad129d24-8775-4701-8450-6f8107dfebeb) service to localhost/127.0.0.1:39847 2023-08-15 15:58:37,655 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data3/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,655 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/cluster_b260da54-e8bc-0c17-f57e-91f8e7e5855f/dfs/data/data4/current/BP-1360484752-172.31.14.131-1692115091062] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:58:37,685 INFO [Listener at localhost/41683] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:58:37,800 INFO [Listener at localhost/41683] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-15 15:58:37,842 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-15 15:58:37,854 INFO [Listener at localhost/41683] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=78 (was 10) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: Listener at localhost/41683 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:39847 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:39847 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:39847 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:39847 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:39847 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@584a08c8 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=467 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=131 (was 139), ProcessCount=169 (was 169), AvailableMemoryMB=947 (was 1686) 2023-08-15 15:58:37,864 INFO [Listener at localhost/41683] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=78, OpenFileDescriptor=467, MaxFileDescriptor=60000, SystemLoadAverage=131, ProcessCount=169, AvailableMemoryMB=947 2023-08-15 15:58:37,864 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-15 15:58:37,864 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/hadoop.log.dir so I do NOT create it in target/test-data/9b19d913-af29-1187-a13e-98560bc02797 2023-08-15 15:58:37,864 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e82c0708-5d2e-9331-170a-4626c3d4fc42/hadoop.tmp.dir so I do NOT create it in target/test-data/9b19d913-af29-1187-a13e-98560bc02797 2023-08-15 15:58:37,864 INFO [Listener at localhost/41683] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/cluster_b40e1168-a372-e19d-5164-059b57279616, deleteOnExit=true 2023-08-15 15:58:37,864 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-15 15:58:37,864 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/test.cache.data in system properties and HBase conf 2023-08-15 15:58:37,864 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/hadoop.tmp.dir in system properties and HBase conf 2023-08-15 15:58:37,865 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/hadoop.log.dir in system properties and HBase conf 2023-08-15 15:58:37,865 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-15 15:58:37,865 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-15 15:58:37,865 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-15 15:58:37,865 DEBUG [Listener at localhost/41683] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-15 15:58:37,866 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-15 15:58:37,866 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-15 15:58:37,866 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-15 15:58:37,866 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-15 15:58:37,867 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-15 15:58:37,867 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-15 15:58:37,867 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-15 15:58:37,867 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-15 15:58:37,867 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-15 15:58:37,868 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/nfs.dump.dir in system properties and HBase conf 2023-08-15 15:58:37,868 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/java.io.tmpdir in system properties and HBase conf 2023-08-15 15:58:37,868 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-15 15:58:37,868 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-15 15:58:37,868 INFO [Listener at localhost/41683] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-15 15:58:37,873 WARN [Listener at localhost/41683] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-15 15:58:37,874 WARN [Listener at localhost/41683] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-15 15:58:37,926 WARN [Listener at localhost/41683] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:37,928 INFO [Listener at localhost/41683] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:37,933 INFO [Listener at localhost/41683] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/java.io.tmpdir/Jetty_localhost_45291_hdfs____7oif6b/webapp 2023-08-15 15:58:38,025 INFO [Listener at localhost/41683] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45291 2023-08-15 15:58:38,029 WARN [Listener at localhost/41683] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-15 15:58:38,030 WARN [Listener at localhost/41683] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-15 15:58:38,069 WARN [Listener at localhost/37617] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:38,083 WARN [Listener at localhost/37617] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:58:38,086 WARN [Listener at localhost/37617] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:38,087 INFO [Listener at localhost/37617] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:38,091 INFO [Listener at localhost/37617] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/java.io.tmpdir/Jetty_localhost_33743_datanode____kx6931/webapp 2023-08-15 15:58:38,183 INFO [Listener at localhost/37617] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33743 2023-08-15 15:58:38,190 WARN [Listener at localhost/35343] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:38,206 WARN [Listener at localhost/35343] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:58:38,208 WARN [Listener at localhost/35343] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:58:38,209 INFO [Listener at localhost/35343] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:58:38,212 INFO [Listener at localhost/35343] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/java.io.tmpdir/Jetty_localhost_33671_datanode____.je05ih/webapp 2023-08-15 15:58:38,294 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3b932b342d347351: Processing first storage report for DS-649f26ed-d818-4eaf-80b8-24835eb4dcda from datanode a0e70078-215c-4e63-97d4-a5783157dc13 2023-08-15 15:58:38,294 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3b932b342d347351: from storage DS-649f26ed-d818-4eaf-80b8-24835eb4dcda node DatanodeRegistration(127.0.0.1:37173, datanodeUuid=a0e70078-215c-4e63-97d4-a5783157dc13, infoPort=40309, infoSecurePort=0, ipcPort=35343, storageInfo=lv=-57;cid=testClusterID;nsid=956223776;c=1692115117876), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:38,294 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3b932b342d347351: Processing first storage report for DS-16075a7a-f359-437a-93e6-50b078c59a67 from datanode a0e70078-215c-4e63-97d4-a5783157dc13 2023-08-15 15:58:38,294 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3b932b342d347351: from storage DS-16075a7a-f359-437a-93e6-50b078c59a67 node DatanodeRegistration(127.0.0.1:37173, datanodeUuid=a0e70078-215c-4e63-97d4-a5783157dc13, infoPort=40309, infoSecurePort=0, ipcPort=35343, storageInfo=lv=-57;cid=testClusterID;nsid=956223776;c=1692115117876), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:38,315 INFO [Listener at localhost/35343] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33671 2023-08-15 15:58:38,323 WARN [Listener at localhost/40391] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:58:38,412 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8f103d87bf01db19: Processing first storage report for DS-2670a3c0-6319-4bed-840d-a5afff00ecb6 from datanode 06ff890a-b1e5-432c-89fd-fd307d3eb110 2023-08-15 15:58:38,412 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8f103d87bf01db19: from storage DS-2670a3c0-6319-4bed-840d-a5afff00ecb6 node DatanodeRegistration(127.0.0.1:43819, datanodeUuid=06ff890a-b1e5-432c-89fd-fd307d3eb110, infoPort=40467, infoSecurePort=0, ipcPort=40391, storageInfo=lv=-57;cid=testClusterID;nsid=956223776;c=1692115117876), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-15 15:58:38,412 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8f103d87bf01db19: Processing first storage report for DS-c788c1f2-1982-4f9d-8b9b-123dd967e5cf from datanode 06ff890a-b1e5-432c-89fd-fd307d3eb110 2023-08-15 15:58:38,412 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8f103d87bf01db19: from storage DS-c788c1f2-1982-4f9d-8b9b-123dd967e5cf node DatanodeRegistration(127.0.0.1:43819, datanodeUuid=06ff890a-b1e5-432c-89fd-fd307d3eb110, infoPort=40467, infoSecurePort=0, ipcPort=40391, storageInfo=lv=-57;cid=testClusterID;nsid=956223776;c=1692115117876), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:58:38,430 DEBUG [Listener at localhost/40391] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797 2023-08-15 15:58:38,433 INFO [Listener at localhost/40391] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/cluster_b40e1168-a372-e19d-5164-059b57279616/zookeeper_0, clientPort=64981, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/cluster_b40e1168-a372-e19d-5164-059b57279616/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/cluster_b40e1168-a372-e19d-5164-059b57279616/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-15 15:58:38,434 INFO [Listener at localhost/40391] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64981 2023-08-15 15:58:38,434 INFO [Listener at localhost/40391] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:38,435 INFO [Listener at localhost/40391] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:38,450 INFO [Listener at localhost/40391] util.FSUtils(471): Created version file at hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d with version=8 2023-08-15 15:58:38,450 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/hbase-staging 2023-08-15 15:58:38,452 INFO [Listener at localhost/40391] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-15 15:58:38,452 INFO [Listener at localhost/40391] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:38,452 INFO [Listener at localhost/40391] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:38,452 INFO [Listener at localhost/40391] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-15 15:58:38,452 INFO [Listener at localhost/40391] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:38,453 INFO [Listener at localhost/40391] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-15 15:58:38,453 INFO [Listener at localhost/40391] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-15 15:58:38,454 INFO [Listener at localhost/40391] ipc.NettyRpcServer(120): Bind to /172.31.14.131:32961 2023-08-15 15:58:38,454 INFO [Listener at localhost/40391] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:38,455 INFO [Listener at localhost/40391] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:38,456 INFO [Listener at localhost/40391] zookeeper.RecoverableZooKeeper(93): Process identifier=master:32961 connecting to ZooKeeper ensemble=127.0.0.1:64981 2023-08-15 15:58:38,462 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:329610x0, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-15 15:58:38,463 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:32961-0x10010aae1070000 connected 2023-08-15 15:58:38,479 DEBUG [Listener at localhost/40391] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 15:58:38,479 DEBUG [Listener at localhost/40391] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:58:38,480 DEBUG [Listener at localhost/40391] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-15 15:58:38,480 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=32961 2023-08-15 15:58:38,480 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=32961 2023-08-15 15:58:38,481 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=32961 2023-08-15 15:58:38,481 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=32961 2023-08-15 15:58:38,481 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=32961 2023-08-15 15:58:38,481 INFO [Listener at localhost/40391] master.HMaster(444): hbase.rootdir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d, hbase.cluster.distributed=false 2023-08-15 15:58:38,494 INFO [Listener at localhost/40391] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-15 15:58:38,494 INFO [Listener at localhost/40391] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:38,494 INFO [Listener at localhost/40391] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:38,494 INFO [Listener at localhost/40391] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-15 15:58:38,494 INFO [Listener at localhost/40391] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:58:38,494 INFO [Listener at localhost/40391] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-15 15:58:38,494 INFO [Listener at localhost/40391] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-15 15:58:38,495 INFO [Listener at localhost/40391] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38177 2023-08-15 15:58:38,496 INFO [Listener at localhost/40391] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-15 15:58:38,497 DEBUG [Listener at localhost/40391] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-15 15:58:38,497 INFO [Listener at localhost/40391] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:38,498 INFO [Listener at localhost/40391] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:38,499 INFO [Listener at localhost/40391] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38177 connecting to ZooKeeper ensemble=127.0.0.1:64981 2023-08-15 15:58:38,503 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:381770x0, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-15 15:58:38,504 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38177-0x10010aae1070001 connected 2023-08-15 15:58:38,504 DEBUG [Listener at localhost/40391] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 15:58:38,505 DEBUG [Listener at localhost/40391] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:58:38,506 DEBUG [Listener at localhost/40391] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-15 15:58:38,509 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38177 2023-08-15 15:58:38,509 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38177 2023-08-15 15:58:38,509 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38177 2023-08-15 15:58:38,509 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38177 2023-08-15 15:58:38,510 DEBUG [Listener at localhost/40391] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38177 2023-08-15 15:58:38,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:58:38,512 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-15 15:58:38,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:58:38,514 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-15 15:58:38,514 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-15 15:58:38,514 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:38,516 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-15 15:58:38,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-15 15:58:38,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,32961,1692115118452 from backup master directory 2023-08-15 15:58:38,519 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:58:38,519 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-15 15:58:38,519 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-15 15:58:38,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:58:38,539 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/hbase.id with ID: c7784a43-e037-4182-b4d1-fdc6e5271dc5 2023-08-15 15:58:38,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:38,559 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:38,569 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x308e9c2f to 127.0.0.1:64981 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:58:38,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@596e2e03, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:58:38,573 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-15 15:58:38,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-15 15:58:38,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:58:38,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store-tmp 2023-08-15 15:58:38,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:38,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-15 15:58:38,585 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:38,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:38,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-15 15:58:38,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:38,585 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:58:38,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 15:58:38,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/WALs/jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:58:38,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C32961%2C1692115118452, suffix=, logDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/WALs/jenkins-hbase4.apache.org,32961,1692115118452, archiveDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/oldWALs, maxLogs=10 2023-08-15 15:58:38,604 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK] 2023-08-15 15:58:38,605 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK] 2023-08-15 15:58:38,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/WALs/jenkins-hbase4.apache.org,32961,1692115118452/jenkins-hbase4.apache.org%2C32961%2C1692115118452.1692115118589 2023-08-15 15:58:38,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK], DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK]] 2023-08-15 15:58:38,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:58:38,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:38,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:38,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:38,611 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:38,612 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-15 15:58:38,613 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-15 15:58:38,614 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:38,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:38,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:38,619 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:58:38,622 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:58:38,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=856209, jitterRate=0.08872608840465546}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:58:38,623 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 15:58:38,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-15 15:58:38,624 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-15 15:58:38,625 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-15 15:58:38,625 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-15 15:58:38,625 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-15 15:58:38,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-15 15:58:38,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-15 15:58:38,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-15 15:58:38,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-15 15:58:38,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-15 15:58:38,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-15 15:58:38,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-15 15:58:38,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-15 15:58:38,641 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-15 15:58:38,643 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:38,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-15 15:58:38,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-15 15:58:38,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-15 15:58:38,649 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-15 15:58:38,649 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:38,649 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-15 15:58:38,649 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,32961,1692115118452, sessionid=0x10010aae1070000, setting cluster-up flag (Was=false) 2023-08-15 15:58:38,653 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:38,659 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-15 15:58:38,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:58:38,663 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:38,668 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-15 15:58:38,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:58:38,670 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.hbase-snapshot/.tmp 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-15 15:58:38,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692115148673 2023-08-15 15:58:38,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-15 15:58:38,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-15 15:58:38,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-15 15:58:38,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-15 15:58:38,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-15 15:58:38,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-15 15:58:38,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-15 15:58:38,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-15 15:58:38,675 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-15 15:58:38,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-15 15:58:38,675 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-15 15:58:38,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-15 15:58:38,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-15 15:58:38,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115118675,5,FailOnTimeoutGroup] 2023-08-15 15:58:38,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115118676,5,FailOnTimeoutGroup] 2023-08-15 15:58:38,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-15 15:58:38,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,677 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-15 15:58:38,692 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-15 15:58:38,693 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-15 15:58:38,693 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d 2023-08-15 15:58:38,703 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:38,704 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-15 15:58:38,706 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/info 2023-08-15 15:58:38,707 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-15 15:58:38,707 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:38,708 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-15 15:58:38,709 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:58:38,709 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-15 15:58:38,710 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:38,710 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-15 15:58:38,712 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(951): ClusterId : c7784a43-e037-4182-b4d1-fdc6e5271dc5 2023-08-15 15:58:38,713 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/table 2023-08-15 15:58:38,713 DEBUG [RS:0;jenkins-hbase4:38177] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-15 15:58:38,714 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-15 15:58:38,715 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:38,716 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740 2023-08-15 15:58:38,716 DEBUG [RS:0;jenkins-hbase4:38177] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-15 15:58:38,717 DEBUG [RS:0;jenkins-hbase4:38177] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-15 15:58:38,717 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740 2023-08-15 15:58:38,720 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-15 15:58:38,721 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-15 15:58:38,723 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:58:38,724 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=759234, jitterRate=-0.03458435833454132}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-15 15:58:38,724 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-15 15:58:38,725 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-15 15:58:38,725 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-15 15:58:38,725 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-15 15:58:38,725 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-15 15:58:38,725 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-15 15:58:38,725 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-15 15:58:38,725 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-15 15:58:38,726 DEBUG [RS:0;jenkins-hbase4:38177] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-15 15:58:38,726 DEBUG [RS:0;jenkins-hbase4:38177] zookeeper.ReadOnlyZKClient(139): Connect 0x63dcc4e0 to 127.0.0.1:64981 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:58:38,728 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-15 15:58:38,728 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-15 15:58:38,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-15 15:58:38,730 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-15 15:58:38,731 DEBUG [RS:0;jenkins-hbase4:38177] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6d1b26eb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:58:38,732 DEBUG [RS:0;jenkins-hbase4:38177] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6924af90, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-15 15:58:38,732 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-15 15:58:38,741 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38177 2023-08-15 15:58:38,742 INFO [RS:0;jenkins-hbase4:38177] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-15 15:58:38,742 INFO [RS:0;jenkins-hbase4:38177] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-15 15:58:38,742 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1022): About to register with Master. 2023-08-15 15:58:38,742 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,32961,1692115118452 with isa=jenkins-hbase4.apache.org/172.31.14.131:38177, startcode=1692115118493 2023-08-15 15:58:38,742 DEBUG [RS:0;jenkins-hbase4:38177] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-15 15:58:38,745 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41677, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-15 15:58:38,746 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=32961] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:38,747 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d 2023-08-15 15:58:38,747 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37617 2023-08-15 15:58:38,747 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-15 15:58:38,748 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 15:58:38,749 DEBUG [RS:0;jenkins-hbase4:38177] zookeeper.ZKUtil(162): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:38,749 WARN [RS:0;jenkins-hbase4:38177] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-15 15:58:38,749 INFO [RS:0;jenkins-hbase4:38177] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:58:38,749 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:38,750 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38177,1692115118493] 2023-08-15 15:58:38,753 DEBUG [RS:0;jenkins-hbase4:38177] zookeeper.ZKUtil(162): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:38,754 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-15 15:58:38,755 INFO [RS:0;jenkins-hbase4:38177] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-15 15:58:38,758 INFO [RS:0;jenkins-hbase4:38177] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-15 15:58:38,758 INFO [RS:0;jenkins-hbase4:38177] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-15 15:58:38,758 INFO [RS:0;jenkins-hbase4:38177] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,758 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-15 15:58:38,759 INFO [RS:0;jenkins-hbase4:38177] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,760 DEBUG [RS:0;jenkins-hbase4:38177] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:58:38,764 INFO [RS:0;jenkins-hbase4:38177] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,764 INFO [RS:0;jenkins-hbase4:38177] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,764 INFO [RS:0;jenkins-hbase4:38177] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,775 INFO [RS:0;jenkins-hbase4:38177] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-15 15:58:38,776 INFO [RS:0;jenkins-hbase4:38177] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38177,1692115118493-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:38,786 INFO [RS:0;jenkins-hbase4:38177] regionserver.Replication(203): jenkins-hbase4.apache.org,38177,1692115118493 started 2023-08-15 15:58:38,786 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38177,1692115118493, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38177, sessionid=0x10010aae1070001 2023-08-15 15:58:38,786 DEBUG [RS:0;jenkins-hbase4:38177] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-15 15:58:38,786 DEBUG [RS:0;jenkins-hbase4:38177] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:38,786 DEBUG [RS:0;jenkins-hbase4:38177] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38177,1692115118493' 2023-08-15 15:58:38,786 DEBUG [RS:0;jenkins-hbase4:38177] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:58:38,787 DEBUG [RS:0;jenkins-hbase4:38177] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:58:38,787 DEBUG [RS:0;jenkins-hbase4:38177] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-15 15:58:38,787 DEBUG [RS:0;jenkins-hbase4:38177] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-15 15:58:38,787 DEBUG [RS:0;jenkins-hbase4:38177] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:38,787 DEBUG [RS:0;jenkins-hbase4:38177] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38177,1692115118493' 2023-08-15 15:58:38,787 DEBUG [RS:0;jenkins-hbase4:38177] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-15 15:58:38,788 DEBUG [RS:0;jenkins-hbase4:38177] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-15 15:58:38,788 DEBUG [RS:0;jenkins-hbase4:38177] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-15 15:58:38,788 INFO [RS:0;jenkins-hbase4:38177] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-15 15:58:38,788 INFO [RS:0;jenkins-hbase4:38177] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-15 15:58:38,882 DEBUG [jenkins-hbase4:32961] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-15 15:58:38,883 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38177,1692115118493, state=OPENING 2023-08-15 15:58:38,885 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-15 15:58:38,887 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:38,887 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38177,1692115118493}] 2023-08-15 15:58:38,887 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-15 15:58:38,891 INFO [RS:0;jenkins-hbase4:38177] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38177%2C1692115118493, suffix=, logDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493, archiveDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/oldWALs, maxLogs=32 2023-08-15 15:58:38,909 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK] 2023-08-15 15:58:38,909 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK] 2023-08-15 15:58:38,912 INFO [RS:0;jenkins-hbase4:38177] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115118893 2023-08-15 15:58:38,912 DEBUG [RS:0;jenkins-hbase4:38177] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK], DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK]] 2023-08-15 15:58:39,042 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:39,043 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-15 15:58:39,045 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50710, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-15 15:58:39,049 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-15 15:58:39,049 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:58:39,051 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38177%2C1692115118493.meta, suffix=.meta, logDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493, archiveDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/oldWALs, maxLogs=32 2023-08-15 15:58:39,066 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK] 2023-08-15 15:58:39,066 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK] 2023-08-15 15:58:39,068 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.meta.1692115119052.meta 2023-08-15 15:58:39,069 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK], DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK]] 2023-08-15 15:58:39,069 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:58:39,069 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-15 15:58:39,069 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-15 15:58:39,070 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-15 15:58:39,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-15 15:58:39,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:39,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-15 15:58:39,070 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-15 15:58:39,072 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-15 15:58:39,073 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/info 2023-08-15 15:58:39,073 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/info 2023-08-15 15:58:39,073 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-15 15:58:39,074 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:39,074 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-15 15:58:39,075 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:58:39,075 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:58:39,075 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-15 15:58:39,076 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:39,076 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-15 15:58:39,077 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/table 2023-08-15 15:58:39,077 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/table 2023-08-15 15:58:39,078 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-15 15:58:39,079 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:39,080 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740 2023-08-15 15:58:39,082 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740 2023-08-15 15:58:39,084 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-15 15:58:39,085 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-15 15:58:39,086 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=863491, jitterRate=0.09798631072044373}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-15 15:58:39,087 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-15 15:58:39,088 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692115119042 2023-08-15 15:58:39,091 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-15 15:58:39,092 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-15 15:58:39,092 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38177,1692115118493, state=OPEN 2023-08-15 15:58:39,094 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-15 15:58:39,094 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-15 15:58:39,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-15 15:58:39,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38177,1692115118493 in 207 msec 2023-08-15 15:58:39,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-15 15:58:39,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 369 msec 2023-08-15 15:58:39,102 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 430 msec 2023-08-15 15:58:39,102 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692115119102, completionTime=-1 2023-08-15 15:58:39,102 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-15 15:58:39,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-15 15:58:39,105 DEBUG [hconnection-0xd64ddca-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-15 15:58:39,107 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50714, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-15 15:58:39,109 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-15 15:58:39,109 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692115179109 2023-08-15 15:58:39,109 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692115239109 2023-08-15 15:58:39,109 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-08-15 15:58:39,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32961,1692115118452-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:39,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32961,1692115118452-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:39,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32961,1692115118452-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:39,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:32961, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:39,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-15 15:58:39,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-15 15:58:39,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-15 15:58:39,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-15 15:58:39,117 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-15 15:58:39,119 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-15 15:58:39,120 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-15 15:58:39,122 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,123 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c empty. 2023-08-15 15:58:39,123 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,123 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-15 15:58:39,138 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-15 15:58:39,140 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 781fc4bf1fa68e5d2f77c9020e47b60c, NAME => 'hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp 2023-08-15 15:58:39,150 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:39,150 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 781fc4bf1fa68e5d2f77c9020e47b60c, disabling compactions & flushes 2023-08-15 15:58:39,150 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:39,150 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:39,150 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. after waiting 0 ms 2023-08-15 15:58:39,150 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:39,150 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:39,150 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 781fc4bf1fa68e5d2f77c9020e47b60c: 2023-08-15 15:58:39,153 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-15 15:58:39,155 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115119154"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115119154"}]},"ts":"1692115119154"} 2023-08-15 15:58:39,157 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-15 15:58:39,158 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-15 15:58:39,158 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115119158"}]},"ts":"1692115119158"} 2023-08-15 15:58:39,160 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-15 15:58:39,168 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=781fc4bf1fa68e5d2f77c9020e47b60c, ASSIGN}] 2023-08-15 15:58:39,170 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=781fc4bf1fa68e5d2f77c9020e47b60c, ASSIGN 2023-08-15 15:58:39,171 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=781fc4bf1fa68e5d2f77c9020e47b60c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38177,1692115118493; forceNewPlan=false, retain=false 2023-08-15 15:58:39,322 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=781fc4bf1fa68e5d2f77c9020e47b60c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:39,322 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115119322"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115119322"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115119322"}]},"ts":"1692115119322"} 2023-08-15 15:58:39,325 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 781fc4bf1fa68e5d2f77c9020e47b60c, server=jenkins-hbase4.apache.org,38177,1692115118493}] 2023-08-15 15:58:39,482 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:39,482 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 781fc4bf1fa68e5d2f77c9020e47b60c, NAME => 'hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:58:39,483 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,483 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:39,483 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,483 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,484 INFO [StoreOpener-781fc4bf1fa68e5d2f77c9020e47b60c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,486 DEBUG [StoreOpener-781fc4bf1fa68e5d2f77c9020e47b60c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c/info 2023-08-15 15:58:39,486 DEBUG [StoreOpener-781fc4bf1fa68e5d2f77c9020e47b60c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c/info 2023-08-15 15:58:39,487 INFO [StoreOpener-781fc4bf1fa68e5d2f77c9020e47b60c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 781fc4bf1fa68e5d2f77c9020e47b60c columnFamilyName info 2023-08-15 15:58:39,487 INFO [StoreOpener-781fc4bf1fa68e5d2f77c9020e47b60c-1] regionserver.HStore(310): Store=781fc4bf1fa68e5d2f77c9020e47b60c/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:39,488 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,489 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,492 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:58:39,494 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:58:39,494 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 781fc4bf1fa68e5d2f77c9020e47b60c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=699984, jitterRate=-0.10992540419101715}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:58:39,494 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 781fc4bf1fa68e5d2f77c9020e47b60c: 2023-08-15 15:58:39,495 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c., pid=6, masterSystemTime=1692115119477 2023-08-15 15:58:39,498 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:39,498 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:39,499 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=781fc4bf1fa68e5d2f77c9020e47b60c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:39,499 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115119499"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115119499"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115119499"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115119499"}]},"ts":"1692115119499"} 2023-08-15 15:58:39,504 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-15 15:58:39,504 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 781fc4bf1fa68e5d2f77c9020e47b60c, server=jenkins-hbase4.apache.org,38177,1692115118493 in 176 msec 2023-08-15 15:58:39,506 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-15 15:58:39,507 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=781fc4bf1fa68e5d2f77c9020e47b60c, ASSIGN in 336 msec 2023-08-15 15:58:39,508 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-15 15:58:39,508 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115119508"}]},"ts":"1692115119508"} 2023-08-15 15:58:39,510 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-15 15:58:39,513 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-15 15:58:39,515 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 397 msec 2023-08-15 15:58:39,518 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-15 15:58:39,519 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:58:39,519 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:39,523 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-15 15:58:39,533 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:58:39,537 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 12 msec 2023-08-15 15:58:39,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-15 15:58:39,554 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:58:39,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 11 msec 2023-08-15 15:58:39,572 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-15 15:58:39,574 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-15 15:58:39,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.055sec 2023-08-15 15:58:39,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-15 15:58:39,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-15 15:58:39,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-15 15:58:39,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32961,1692115118452-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-15 15:58:39,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32961,1692115118452-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-15 15:58:39,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-15 15:58:39,612 DEBUG [Listener at localhost/40391] zookeeper.ReadOnlyZKClient(139): Connect 0x0dd2e70b to 127.0.0.1:64981 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:58:39,617 DEBUG [Listener at localhost/40391] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2e108013, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:58:39,619 DEBUG [hconnection-0x7a0165a3-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-15 15:58:39,621 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50716, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-15 15:58:39,623 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:58:39,624 INFO [Listener at localhost/40391] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:58:39,627 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-15 15:58:39,627 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:58:39,628 INFO [Listener at localhost/40391] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-15 15:58:39,630 DEBUG [Listener at localhost/40391] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-15 15:58:39,633 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47056, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-15 15:58:39,635 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-15 15:58:39,635 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-15 15:58:39,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-15 15:58:39,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:39,640 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-08-15 15:58:39,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testCompactionRecordDoesntBlockRolling" procId is: 9 2023-08-15 15:58:39,641 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-15 15:58:39,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-15 15:58:39,643 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:39,644 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1 empty. 2023-08-15 15:58:39,644 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:39,644 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testCompactionRecordDoesntBlockRolling regions 2023-08-15 15:58:39,666 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/.tabledesc/.tableinfo.0000000001 2023-08-15 15:58:39,668 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 47a9f5e363dc79498dd5c5fb6d4fffd1, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/.tmp 2023-08-15 15:58:39,680 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:39,680 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1604): Closing 47a9f5e363dc79498dd5c5fb6d4fffd1, disabling compactions & flushes 2023-08-15 15:58:39,680 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:39,680 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:39,681 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. after waiting 0 ms 2023-08-15 15:58:39,681 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:39,681 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:39,681 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1558): Region close journal for 47a9f5e363dc79498dd5c5fb6d4fffd1: 2023-08-15 15:58:39,684 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ADD_TO_META 2023-08-15 15:58:39,685 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1692115119685"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115119685"}]},"ts":"1692115119685"} 2023-08-15 15:58:39,688 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-15 15:58:39,689 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-15 15:58:39,689 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115119689"}]},"ts":"1692115119689"} 2023-08-15 15:58:39,691 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLING in hbase:meta 2023-08-15 15:58:39,700 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=47a9f5e363dc79498dd5c5fb6d4fffd1, ASSIGN}] 2023-08-15 15:58:39,701 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=47a9f5e363dc79498dd5c5fb6d4fffd1, ASSIGN 2023-08-15 15:58:39,702 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=47a9f5e363dc79498dd5c5fb6d4fffd1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38177,1692115118493; forceNewPlan=false, retain=false 2023-08-15 15:58:39,854 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=47a9f5e363dc79498dd5c5fb6d4fffd1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:39,854 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1692115119853"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115119853"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115119853"}]},"ts":"1692115119853"} 2023-08-15 15:58:39,857 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 47a9f5e363dc79498dd5c5fb6d4fffd1, server=jenkins-hbase4.apache.org,38177,1692115118493}] 2023-08-15 15:58:40,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:40,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 47a9f5e363dc79498dd5c5fb6d4fffd1, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:58:40,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testCompactionRecordDoesntBlockRolling 47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:40,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:58:40,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:40,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:40,017 INFO [StoreOpener-47a9f5e363dc79498dd5c5fb6d4fffd1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:40,018 DEBUG [StoreOpener-47a9f5e363dc79498dd5c5fb6d4fffd1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info 2023-08-15 15:58:40,018 DEBUG [StoreOpener-47a9f5e363dc79498dd5c5fb6d4fffd1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info 2023-08-15 15:58:40,019 INFO [StoreOpener-47a9f5e363dc79498dd5c5fb6d4fffd1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47a9f5e363dc79498dd5c5fb6d4fffd1 columnFamilyName info 2023-08-15 15:58:40,019 INFO [StoreOpener-47a9f5e363dc79498dd5c5fb6d4fffd1-1] regionserver.HStore(310): Store=47a9f5e363dc79498dd5c5fb6d4fffd1/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:58:40,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:40,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:40,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:58:40,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:58:40,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 47a9f5e363dc79498dd5c5fb6d4fffd1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=813516, jitterRate=0.03443944454193115}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:58:40,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 47a9f5e363dc79498dd5c5fb6d4fffd1: 2023-08-15 15:58:40,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1., pid=11, masterSystemTime=1692115120010 2023-08-15 15:58:40,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:40,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:40,032 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=47a9f5e363dc79498dd5c5fb6d4fffd1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:40,032 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1692115120031"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115120031"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115120031"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115120031"}]},"ts":"1692115120031"} 2023-08-15 15:58:40,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-15 15:58:40,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 47a9f5e363dc79498dd5c5fb6d4fffd1, server=jenkins-hbase4.apache.org,38177,1692115118493 in 177 msec 2023-08-15 15:58:40,040 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-15 15:58:40,040 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=47a9f5e363dc79498dd5c5fb6d4fffd1, ASSIGN in 337 msec 2023-08-15 15:58:40,041 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-15 15:58:40,042 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115120041"}]},"ts":"1692115120041"} 2023-08-15 15:58:40,043 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLED in hbase:meta 2023-08-15 15:58:40,046 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_POST_OPERATION 2023-08-15 15:58:40,048 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling in 410 msec 2023-08-15 15:58:40,127 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-15 15:58:44,253 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-15 15:58:44,253 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-15 15:58:44,755 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-15 15:58:44,755 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-15 15:58:44,756 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:58:45,518 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-15 15:58:49,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-15 15:58:49,644 INFO [Listener at localhost/40391] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testCompactionRecordDoesntBlockRolling, procId: 9 completed 2023-08-15 15:58:49,647 DEBUG [Listener at localhost/40391] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:49,647 DEBUG [Listener at localhost/40391] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:49,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-15 15:58:49,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure.ProcedureCoordinator(165): Submitting procedure hbase:namespace 2023-08-15 15:58:49,666 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'hbase:namespace' 2023-08-15 15:58:49,666 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:58:49,667 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'hbase:namespace' starting 'acquire' 2023-08-15 15:58:49,667 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'hbase:namespace', kicking off acquire phase on members. 2023-08-15 15:58:49,668 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,668 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-15 15:58:49,669 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:58:49,669 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:58:49,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:58:49,669 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,670 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-15 15:58:49,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/hbase:namespace 2023-08-15 15:58:49,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-15 15:58:49,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-15 15:58:49,671 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for hbase:namespace 2023-08-15 15:58:49,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:hbase:namespace 2023-08-15 15:58:49,673 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'hbase:namespace' with timeout 60000ms 2023-08-15 15:58:49,673 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:58:49,673 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'hbase:namespace' starting 'acquire' stage 2023-08-15 15:58:49,674 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-15 15:58:49,674 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-15 15:58:49,674 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:49,674 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. started... 2023-08-15 15:58:49,674 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 781fc4bf1fa68e5d2f77c9020e47b60c 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-15 15:58:49,687 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c/.tmp/info/f3937bd89eb24461900037de38226367 2023-08-15 15:58:49,695 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c/.tmp/info/f3937bd89eb24461900037de38226367 as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c/info/f3937bd89eb24461900037de38226367 2023-08-15 15:58:49,702 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c/info/f3937bd89eb24461900037de38226367, entries=2, sequenceid=6, filesize=4.8 K 2023-08-15 15:58:49,703 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 781fc4bf1fa68e5d2f77c9020e47b60c in 29ms, sequenceid=6, compaction requested=false 2023-08-15 15:58:49,704 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 781fc4bf1fa68e5d2f77c9020e47b60c: 2023-08-15 15:58:49,704 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:58:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-15 15:58:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-15 15:58:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'hbase:namespace' locally acquired 2023-08-15 15:58:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure (hbase:namespace) in zk 2023-08-15 15:58:49,706 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,706 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-15 15:58:49,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:58:49,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:58:49,707 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace 2023-08-15 15:58:49,707 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'hbase:namespace' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-15 15:58:49,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:58:49,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:58:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-15 15:58:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:58:49,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure 'hbase:namespace' on coordinator 2023-08-15 15:58:49,709 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'hbase:namespace' starting 'in-barrier' execution. 2023-08-15 15:58:49,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@311824d7[Count = 0] remaining members to acquire global barrier 2023-08-15 15:58:49,709 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-15 15:58:49,711 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-15 15:58:49,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-15 15:58:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/hbase:namespace 2023-08-15 15:58:49,712 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'hbase:namespace' received 'reached' from coordinator. 2023-08-15 15:58:49,712 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'hbase:namespace' locally completed 2023-08-15 15:58:49,712 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'hbase:namespace' completed for member 'jenkins-hbase4.apache.org,38177,1692115118493' in zk 2023-08-15 15:58:49,712 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,712 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-15 15:58:49,714 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'hbase:namespace' has notified controller of completion 2023-08-15 15:58:49,714 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,714 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:58:49,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:58:49,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:58:49,714 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'hbase:namespace' completed. 2023-08-15 15:58:49,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:58:49,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:58:49,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-15 15:58:49,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:58:49,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-15 15:58:49,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'hbase:namespace' member 'jenkins-hbase4.apache.org,38177,1692115118493': 2023-08-15 15:58:49,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' released barrier for procedure'hbase:namespace', counting down latch. Waiting for 0 more 2023-08-15 15:58:49,718 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'hbase:namespace' execution completed 2023-08-15 15:58:49,718 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-15 15:58:49,718 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-15 15:58:49,718 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:hbase:namespace 2023-08-15 15:58:49,718 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure hbase:namespaceincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-15 15:58:49,720 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,720 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,721 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:58:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:58:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:58:49,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:58:49,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:58:49,722 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:58:49,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-15 15:58:49,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:58:49,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-15 15:58:49,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,723 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:58:49,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-08-15 15:58:49,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,729 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,729 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:58:49,729 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-15 15:58:49,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:58:49,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:58:49,729 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-08-15 15:58:49,729 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:58:49,730 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:49,730 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:58:49,730 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-15 15:58:49,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'hbase:namespace' 2023-08-15 15:58:49,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:58:49,730 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-08-15 15:58:49,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:58:49,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-15 15:58:49,731 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-08-15 15:58:49,733 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : hbase:namespace'' to complete. (max 20000 ms per retry) 2023-08-15 15:58:49,733 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-15 15:58:59,733 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-15 15:58:59,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-15 15:58:59,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-15 15:58:59,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,754 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:58:59,754 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:58:59,754 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-15 15:58:59,754 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-15 15:58:59,755 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,755 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,758 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,758 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:58:59,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:58:59,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:58:59,758 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,758 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-15 15:58:59,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-15 15:58:59,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,759 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-15 15:58:59,759 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:58:59,760 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-15 15:58:59,760 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-15 15:58:59,760 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-15 15:58:59,760 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:59,760 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. started... 2023-08-15 15:58:59,760 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 47a9f5e363dc79498dd5c5fb6d4fffd1 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-15 15:58:59,780 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=5 (bloomFilter=true), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/5efb67f181684723afb5f41669d303a3 2023-08-15 15:58:59,790 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/5efb67f181684723afb5f41669d303a3 as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/5efb67f181684723afb5f41669d303a3 2023-08-15 15:58:59,797 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/5efb67f181684723afb5f41669d303a3, entries=1, sequenceid=5, filesize=5.8 K 2023-08-15 15:58:59,798 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 47a9f5e363dc79498dd5c5fb6d4fffd1 in 38ms, sequenceid=5, compaction requested=false 2023-08-15 15:58:59,798 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 47a9f5e363dc79498dd5c5fb6d4fffd1: 2023-08-15 15:58:59,798 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:58:59,799 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-15 15:58:59,799 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-15 15:58:59,799 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,799 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-15 15:58:59,799 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-15 15:58:59,802 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,802 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:58:59,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:58:59,802 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,802 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-15 15:58:59,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:58:59,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:58:59,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:58:59,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-15 15:58:59,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3527a62c[Count = 0] remaining members to acquire global barrier 2023-08-15 15:58:59,804 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-15 15:58:59,804 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,805 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,806 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-15 15:58:59,806 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-15 15:58:59,806 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,38177,1692115118493' in zk 2023-08-15 15:58:59,806 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,806 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-15 15:58:59,807 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-15 15:58:59,807 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,807 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:58:59,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:58:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:58:59,807 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-15 15:58:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:58:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:58:59,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:58:59,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,38177,1692115118493': 2023-08-15 15:58:59,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-15 15:58:59,811 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-15 15:58:59,811 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-15 15:58:59,811 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-15 15:58:59,811 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,811 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-15 15:58:59,816 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,816 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:58:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:58:59,816 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:58:59,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:58:59,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:58:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:58:59,817 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:58:59,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,819 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:58:59,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,821 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,821 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,821 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:58:59,821 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:58:59,822 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:58:59,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:58:59,822 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-15 15:58:59,822 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:58:59,822 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:58:59,822 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-15 15:58:59,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:58:59,822 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,822 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:58:59,823 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-15 15:58:59,823 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-15 15:58:59,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:58:59,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:59:09,823 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-15 15:59:09,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-15 15:59:09,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-15 15:59:09,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-15 15:59:09,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,834 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:59:09,834 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:59:09,835 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-15 15:59:09,835 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-15 15:59:09,835 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,835 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,837 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,837 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:59:09,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:59:09,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:59:09,837 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,837 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-15 15:59:09,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,838 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-15 15:59:09,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,838 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,838 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-15 15:59:09,838 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,839 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-15 15:59:09,839 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:59:09,839 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-15 15:59:09,839 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-15 15:59:09,839 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-15 15:59:09,839 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:09,839 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. started... 2023-08-15 15:59:09,839 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 47a9f5e363dc79498dd5c5fb6d4fffd1 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-15 15:59:09,852 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/1a4af58774854196b2432dda774e9adf 2023-08-15 15:59:09,861 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/1a4af58774854196b2432dda774e9adf as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/1a4af58774854196b2432dda774e9adf 2023-08-15 15:59:09,868 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/1a4af58774854196b2432dda774e9adf, entries=1, sequenceid=9, filesize=5.8 K 2023-08-15 15:59:09,869 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 47a9f5e363dc79498dd5c5fb6d4fffd1 in 30ms, sequenceid=9, compaction requested=false 2023-08-15 15:59:09,869 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 47a9f5e363dc79498dd5c5fb6d4fffd1: 2023-08-15 15:59:09,869 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:09,869 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-15 15:59:09,869 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-15 15:59:09,869 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,869 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-15 15:59:09,870 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-15 15:59:09,871 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,871 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:09,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:09,872 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,872 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-15 15:59:09,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:09,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:09,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:09,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-15 15:59:09,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70252eec[Count = 0] remaining members to acquire global barrier 2023-08-15 15:59:09,874 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-15 15:59:09,874 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,876 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,876 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-15 15:59:09,876 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,876 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-15 15:59:09,876 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-15 15:59:09,876 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,38177,1692115118493' in zk 2023-08-15 15:59:09,878 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-15 15:59:09,878 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,878 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:59:09,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:09,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:09,878 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-15 15:59:09,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:09,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:09,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:09,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,38177,1692115118493': 2023-08-15 15:59:09,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-15 15:59:09,881 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-15 15:59:09,881 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-15 15:59:09,881 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-15 15:59:09,881 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,881 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-15 15:59:09,883 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,883 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:09,883 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:59:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:09,884 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:59:09,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:59:09,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:09,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,885 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:09,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,888 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,888 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,888 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:59:09,888 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:59:09,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-15 15:59:09,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:59:09,888 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:59:09,888 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:59:09,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:59:09,888 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:09,889 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,889 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-15 15:59:09,889 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-15 15:59:09,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:59:09,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:59:09,889 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:09,889 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,889 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-15 15:59:19,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-15 15:59:19,909 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK] 2023-08-15 15:59:19,910 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK] 2023-08-15 15:59:19,913 INFO [Listener at localhost/40391] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115118893 with entries=13, filesize=6.36 KB; new WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115159893 2023-08-15 15:59:19,913 DEBUG [Listener at localhost/40391] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK], DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK]] 2023-08-15 15:59:19,913 DEBUG [Listener at localhost/40391] wal.AbstractFSWAL(716): hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115118893 is not closed yet, will try archiving it next time 2023-08-15 15:59:19,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-15 15:59:19,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-15 15:59:19,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,921 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:59:19,921 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:59:19,921 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-15 15:59:19,921 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-15 15:59:19,922 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,922 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,923 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:59:19,923 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:59:19,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:59:19,924 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,924 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-15 15:59:19,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-15 15:59:19,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-15 15:59:19,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,925 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-15 15:59:19,925 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:59:19,925 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-15 15:59:19,925 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-15 15:59:19,926 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-15 15:59:19,926 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:19,926 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. started... 2023-08-15 15:59:19,926 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 47a9f5e363dc79498dd5c5fb6d4fffd1 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-15 15:59:19,940 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/2de351b095884078b81afe0ce360e8db 2023-08-15 15:59:19,948 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/2de351b095884078b81afe0ce360e8db as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/2de351b095884078b81afe0ce360e8db 2023-08-15 15:59:19,956 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/2de351b095884078b81afe0ce360e8db, entries=1, sequenceid=13, filesize=5.8 K 2023-08-15 15:59:19,957 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 47a9f5e363dc79498dd5c5fb6d4fffd1 in 31ms, sequenceid=13, compaction requested=true 2023-08-15 15:59:19,957 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 47a9f5e363dc79498dd5c5fb6d4fffd1: 2023-08-15 15:59:19,957 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:19,957 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-15 15:59:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-15 15:59:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-15 15:59:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-15 15:59:19,960 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,960 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:19,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:19,960 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,960 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-15 15:59:19,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:19,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:19,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:19,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-15 15:59:19,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31d1b899[Count = 0] remaining members to acquire global barrier 2023-08-15 15:59:19,962 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-15 15:59:19,962 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,963 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,963 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-15 15:59:19,963 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-15 15:59:19,963 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,963 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-15 15:59:19,963 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,38177,1692115118493' in zk 2023-08-15 15:59:19,966 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-15 15:59:19,966 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,966 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:59:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:19,966 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-15 15:59:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:19,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,38177,1692115118493': 2023-08-15 15:59:19,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-15 15:59:19,969 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-15 15:59:19,969 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-15 15:59:19,969 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-15 15:59:19,969 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,969 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-15 15:59:19,970 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,970 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:19,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,970 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:59:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:19,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:59:19,971 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:59:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:19,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,972 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,975 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,975 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:59:19,975 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:59:19,975 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:59:19,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:59:19,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-15 15:59:19,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:59:19,975 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:59:19,975 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,976 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:19,976 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,976 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,976 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:19,976 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-15 15:59:19,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:59:19,976 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-15 15:59:19,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:59:29,976 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-15 15:59:29,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-15 15:59:29,979 DEBUG [Listener at localhost/40391] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 15:59:29,984 DEBUG [Listener at localhost/40391] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 17694 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 15:59:29,986 DEBUG [Listener at localhost/40391] regionserver.HStore(1912): 47a9f5e363dc79498dd5c5fb6d4fffd1/info is initiating minor compaction (all files) 2023-08-15 15:59:29,986 INFO [Listener at localhost/40391] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-15 15:59:29,986 INFO [Listener at localhost/40391] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:29,986 INFO [Listener at localhost/40391] regionserver.HRegion(2259): Starting compaction of 47a9f5e363dc79498dd5c5fb6d4fffd1/info in TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:29,986 INFO [Listener at localhost/40391] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/5efb67f181684723afb5f41669d303a3, hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/1a4af58774854196b2432dda774e9adf, hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/2de351b095884078b81afe0ce360e8db] into tmpdir=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp, totalSize=17.3 K 2023-08-15 15:59:29,987 DEBUG [Listener at localhost/40391] compactions.Compactor(207): Compacting 5efb67f181684723afb5f41669d303a3, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1692115139743 2023-08-15 15:59:29,988 DEBUG [Listener at localhost/40391] compactions.Compactor(207): Compacting 1a4af58774854196b2432dda774e9adf, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1692115149826 2023-08-15 15:59:29,988 DEBUG [Listener at localhost/40391] compactions.Compactor(207): Compacting 2de351b095884078b81afe0ce360e8db, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1692115159891 2023-08-15 15:59:30,009 INFO [Listener at localhost/40391] throttle.PressureAwareThroughputController(145): 47a9f5e363dc79498dd5c5fb6d4fffd1#info#compaction#9 average throughput is 3.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 15:59:30,029 DEBUG [Listener at localhost/40391] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/9642ec942dcd48fbb1cb78dc052b73b9 as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/9642ec942dcd48fbb1cb78dc052b73b9 2023-08-15 15:59:30,046 INFO [Listener at localhost/40391] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 47a9f5e363dc79498dd5c5fb6d4fffd1/info of 47a9f5e363dc79498dd5c5fb6d4fffd1 into 9642ec942dcd48fbb1cb78dc052b73b9(size=7.9 K), total size for store is 7.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 15:59:30,046 DEBUG [Listener at localhost/40391] regionserver.HRegion(2289): Compaction status journal for 47a9f5e363dc79498dd5c5fb6d4fffd1: 2023-08-15 15:59:30,064 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK] 2023-08-15 15:59:30,064 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK] 2023-08-15 15:59:30,066 INFO [Listener at localhost/40391] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115159893 with entries=4, filesize=2.43 KB; new WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115170048 2023-08-15 15:59:30,067 DEBUG [Listener at localhost/40391] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK], DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK]] 2023-08-15 15:59:30,067 DEBUG [Listener at localhost/40391] wal.AbstractFSWAL(716): hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115159893 is not closed yet, will try archiving it next time 2023-08-15 15:59:30,067 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115118893 to hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/oldWALs/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115118893 2023-08-15 15:59:30,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-15 15:59:30,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-08-15 15:59:30,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,076 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:59:30,076 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:59:30,077 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-08-15 15:59:30,077 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-08-15 15:59:30,078 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,078 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,083 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,083 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:59:30,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:59:30,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:59:30,083 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,083 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-15 15:59:30,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-15 15:59:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,084 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-08-15 15:59:30,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,084 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-08-15 15:59:30,084 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-15 15:59:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-08-15 15:59:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-15 15:59:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-15 15:59:30,085 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:30,085 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. started... 2023-08-15 15:59:30,085 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 47a9f5e363dc79498dd5c5fb6d4fffd1 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-15 15:59:30,102 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=18 (bloomFilter=true), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/ed886cd53e724eb38b0f90f68cc35c05 2023-08-15 15:59:30,111 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/ed886cd53e724eb38b0f90f68cc35c05 as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/ed886cd53e724eb38b0f90f68cc35c05 2023-08-15 15:59:30,118 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/ed886cd53e724eb38b0f90f68cc35c05, entries=1, sequenceid=18, filesize=5.8 K 2023-08-15 15:59:30,119 INFO [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 47a9f5e363dc79498dd5c5fb6d4fffd1 in 34ms, sequenceid=18, compaction requested=false 2023-08-15 15:59:30,119 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 47a9f5e363dc79498dd5c5fb6d4fffd1: 2023-08-15 15:59:30,120 DEBUG [rs(jenkins-hbase4.apache.org,38177,1692115118493)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-15 15:59:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-15 15:59:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-08-15 15:59:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-08-15 15:59:30,122 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,122 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:30,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-15 15:59:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:30,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38177,1692115118493' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-08-15 15:59:30,125 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@411f1ce8[Count = 0] remaining members to acquire global barrier 2023-08-15 15:59:30,125 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-08-15 15:59:30,125 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,126 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,126 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-08-15 15:59:30,127 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-08-15 15:59:30,127 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,38177,1692115118493' in zk 2023-08-15 15:59:30,127 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,127 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-15 15:59:30,128 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-08-15 15:59:30,128 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,128 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:59:30,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:30,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:30,129 DEBUG [member: 'jenkins-hbase4.apache.org,38177,1692115118493' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-08-15 15:59:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:30,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,38177,1692115118493': 2023-08-15 15:59:30,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38177,1692115118493' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-08-15 15:59:30,132 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-08-15 15:59:30,132 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-15 15:59:30,132 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-15 15:59:30,132 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,132 INFO [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-15 15:59:30,135 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,135 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-15 15:59:30,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-15 15:59:30,135 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:59:30,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-15 15:59:30,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:59:30,136 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:59:30,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-15 15:59:30,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,137 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-15 15:59:30,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,140 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-15 15:59:30,140 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-15 15:59:30,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-08-15 15:59:30,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-15 15:59:30,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:59:30,140 DEBUG [(jenkins-hbase4.apache.org,32961,1692115118452)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-15 15:59:30,140 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,140 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-15 15:59:30,141 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,141 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-08-15 15:59:30,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-08-15 15:59:30,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-15 15:59:30,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:59:30,141 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:30,142 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-08-15 15:59:30,142 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,142 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:30,142 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-08-15 15:59:40,142 DEBUG [Listener at localhost/40391] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-15 15:59:40,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32961] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-15 15:59:40,164 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK] 2023-08-15 15:59:40,164 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK] 2023-08-15 15:59:40,169 INFO [Listener at localhost/40391] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115170048 with entries=3, filesize=1.94 KB; new WAL /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115180147 2023-08-15 15:59:40,170 DEBUG [Listener at localhost/40391] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43819,DS-2670a3c0-6319-4bed-840d-a5afff00ecb6,DISK], DatanodeInfoWithStorage[127.0.0.1:37173,DS-649f26ed-d818-4eaf-80b8-24835eb4dcda,DISK]] 2023-08-15 15:59:40,170 DEBUG [Listener at localhost/40391] wal.AbstractFSWAL(716): hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115170048 is not closed yet, will try archiving it next time 2023-08-15 15:59:40,170 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-15 15:59:40,170 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/WALs/jenkins-hbase4.apache.org,38177,1692115118493/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115159893 to hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/oldWALs/jenkins-hbase4.apache.org%2C38177%2C1692115118493.1692115159893 2023-08-15 15:59:40,170 INFO [Listener at localhost/40391] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-15 15:59:40,170 DEBUG [Listener at localhost/40391] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0dd2e70b to 127.0.0.1:64981 2023-08-15 15:59:40,171 DEBUG [Listener at localhost/40391] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:59:40,171 DEBUG [Listener at localhost/40391] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-15 15:59:40,171 DEBUG [Listener at localhost/40391] util.JVMClusterUtil(257): Found active master hash=117200038, stopped=false 2023-08-15 15:59:40,171 INFO [Listener at localhost/40391] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:59:40,175 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-15 15:59:40,175 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-15 15:59:40,175 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:40,175 INFO [Listener at localhost/40391] procedure2.ProcedureExecutor(629): Stopping 2023-08-15 15:59:40,176 DEBUG [Listener at localhost/40391] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x308e9c2f to 127.0.0.1:64981 2023-08-15 15:59:40,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:59:40,176 DEBUG [Listener at localhost/40391] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:59:40,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:59:40,176 INFO [Listener at localhost/40391] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38177,1692115118493' ***** 2023-08-15 15:59:40,176 INFO [Listener at localhost/40391] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-15 15:59:40,176 INFO [RS:0;jenkins-hbase4:38177] regionserver.HeapMemoryManager(220): Stopping 2023-08-15 15:59:40,177 INFO [RS:0;jenkins-hbase4:38177] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-15 15:59:40,177 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-15 15:59:40,177 INFO [RS:0;jenkins-hbase4:38177] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-15 15:59:40,177 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(3305): Received CLOSE for 781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:59:40,177 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(3305): Received CLOSE for 47a9f5e363dc79498dd5c5fb6d4fffd1 2023-08-15 15:59:40,178 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:40,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 781fc4bf1fa68e5d2f77c9020e47b60c, disabling compactions & flushes 2023-08-15 15:59:40,178 DEBUG [RS:0;jenkins-hbase4:38177] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x63dcc4e0 to 127.0.0.1:64981 2023-08-15 15:59:40,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:59:40,178 DEBUG [RS:0;jenkins-hbase4:38177] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:59:40,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:59:40,178 INFO [RS:0;jenkins-hbase4:38177] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-15 15:59:40,178 INFO [RS:0;jenkins-hbase4:38177] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-15 15:59:40,178 INFO [RS:0;jenkins-hbase4:38177] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-15 15:59:40,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. after waiting 0 ms 2023-08-15 15:59:40,178 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-15 15:59:40,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:59:40,178 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-08-15 15:59:40,178 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1478): Online Regions={781fc4bf1fa68e5d2f77c9020e47b60c=hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c., 47a9f5e363dc79498dd5c5fb6d4fffd1=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1., 1588230740=hbase:meta,,1.1588230740} 2023-08-15 15:59:40,178 DEBUG [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1504): Waiting on 1588230740, 47a9f5e363dc79498dd5c5fb6d4fffd1, 781fc4bf1fa68e5d2f77c9020e47b60c 2023-08-15 15:59:40,183 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-15 15:59:40,183 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-15 15:59:40,183 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-15 15:59:40,183 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-15 15:59:40,183 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-15 15:59:40,183 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.10 KB heapSize=5.61 KB 2023-08-15 15:59:40,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/namespace/781fc4bf1fa68e5d2f77c9020e47b60c/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-15 15:59:40,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:59:40,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 781fc4bf1fa68e5d2f77c9020e47b60c: 2023-08-15 15:59:40,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692115119115.781fc4bf1fa68e5d2f77c9020e47b60c. 2023-08-15 15:59:40,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 47a9f5e363dc79498dd5c5fb6d4fffd1, disabling compactions & flushes 2023-08-15 15:59:40,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:40,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:40,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. after waiting 0 ms 2023-08-15 15:59:40,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:40,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 47a9f5e363dc79498dd5c5fb6d4fffd1 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-08-15 15:59:40,200 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.84 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/.tmp/info/95c533b9938a4140a4dd0c95fd0d239c 2023-08-15 15:59:40,213 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/65e17b12fe7d4760bc224513c0d0be09 2023-08-15 15:59:40,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/.tmp/info/65e17b12fe7d4760bc224513c0d0be09 as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/65e17b12fe7d4760bc224513c0d0be09 2023-08-15 15:59:40,300 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=264 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/.tmp/table/11f8b3aa2efc4b81a20133d7c68ab62f 2023-08-15 15:59:40,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/65e17b12fe7d4760bc224513c0d0be09, entries=1, sequenceid=22, filesize=5.8 K 2023-08-15 15:59:40,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for 47a9f5e363dc79498dd5c5fb6d4fffd1 in 116ms, sequenceid=22, compaction requested=true 2023-08-15 15:59:40,311 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/.tmp/info/95c533b9938a4140a4dd0c95fd0d239c as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/info/95c533b9938a4140a4dd0c95fd0d239c 2023-08-15 15:59:40,315 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/5efb67f181684723afb5f41669d303a3, hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/1a4af58774854196b2432dda774e9adf, hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/2de351b095884078b81afe0ce360e8db] to archive 2023-08-15 15:59:40,316 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-15 15:59:40,320 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/5efb67f181684723afb5f41669d303a3 to hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/5efb67f181684723afb5f41669d303a3 2023-08-15 15:59:40,320 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/info/95c533b9938a4140a4dd0c95fd0d239c, entries=20, sequenceid=14, filesize=7.6 K 2023-08-15 15:59:40,323 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/1a4af58774854196b2432dda774e9adf to hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/1a4af58774854196b2432dda774e9adf 2023-08-15 15:59:40,325 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/.tmp/table/11f8b3aa2efc4b81a20133d7c68ab62f as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/table/11f8b3aa2efc4b81a20133d7c68ab62f 2023-08-15 15:59:40,325 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/2de351b095884078b81afe0ce360e8db to hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/info/2de351b095884078b81afe0ce360e8db 2023-08-15 15:59:40,337 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/table/11f8b3aa2efc4b81a20133d7c68ab62f, entries=4, sequenceid=14, filesize=4.9 K 2023-08-15 15:59:40,339 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.10 KB/3174, heapSize ~5.33 KB/5456, currentSize=0 B/0 for 1588230740 in 156ms, sequenceid=14, compaction requested=false 2023-08-15 15:59:40,353 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-08-15 15:59:40,356 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-15 15:59:40,356 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-15 15:59:40,357 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-15 15:59:40,357 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-15 15:59:40,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/47a9f5e363dc79498dd5c5fb6d4fffd1/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=1 2023-08-15 15:59:40,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:40,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 47a9f5e363dc79498dd5c5fb6d4fffd1: 2023-08-15 15:59:40,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1692115119635.47a9f5e363dc79498dd5c5fb6d4fffd1. 2023-08-15 15:59:40,378 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38177,1692115118493; all regions closed. 2023-08-15 15:59:40,384 DEBUG [RS:0;jenkins-hbase4:38177] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/oldWALs 2023-08-15 15:59:40,384 INFO [RS:0;jenkins-hbase4:38177] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38177%2C1692115118493.meta:.meta(num 1692115119052) 2023-08-15 15:59:40,395 DEBUG [RS:0;jenkins-hbase4:38177] wal.AbstractFSWAL(1028): Moved 2 WAL file(s) to /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/oldWALs 2023-08-15 15:59:40,395 INFO [RS:0;jenkins-hbase4:38177] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38177%2C1692115118493:(num 1692115180147) 2023-08-15 15:59:40,395 DEBUG [RS:0;jenkins-hbase4:38177] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:59:40,395 INFO [RS:0;jenkins-hbase4:38177] regionserver.LeaseManager(133): Closed leases 2023-08-15 15:59:40,396 INFO [RS:0;jenkins-hbase4:38177] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-15 15:59:40,396 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-15 15:59:40,397 INFO [RS:0;jenkins-hbase4:38177] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38177 2023-08-15 15:59:40,401 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 15:59:40,401 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38177,1692115118493 2023-08-15 15:59:40,401 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 15:59:40,403 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38177,1692115118493] 2023-08-15 15:59:40,403 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38177,1692115118493; numProcessing=1 2023-08-15 15:59:40,405 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38177,1692115118493 already deleted, retry=false 2023-08-15 15:59:40,405 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38177,1692115118493 expired; onlineServers=0 2023-08-15 15:59:40,405 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,32961,1692115118452' ***** 2023-08-15 15:59:40,405 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-15 15:59:40,406 DEBUG [M:0;jenkins-hbase4:32961] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c3c0a1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-15 15:59:40,406 INFO [M:0;jenkins-hbase4:32961] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,32961,1692115118452 2023-08-15 15:59:40,406 INFO [M:0;jenkins-hbase4:32961] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,32961,1692115118452; all regions closed. 2023-08-15 15:59:40,406 DEBUG [M:0;jenkins-hbase4:32961] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 15:59:40,406 DEBUG [M:0;jenkins-hbase4:32961] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-15 15:59:40,406 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-15 15:59:40,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115118675] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115118675,5,FailOnTimeoutGroup] 2023-08-15 15:59:40,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115118676] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115118676,5,FailOnTimeoutGroup] 2023-08-15 15:59:40,406 DEBUG [M:0;jenkins-hbase4:32961] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-15 15:59:40,407 INFO [M:0;jenkins-hbase4:32961] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-15 15:59:40,407 INFO [M:0;jenkins-hbase4:32961] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-15 15:59:40,407 INFO [M:0;jenkins-hbase4:32961] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-15 15:59:40,407 DEBUG [M:0;jenkins-hbase4:32961] master.HMaster(1512): Stopping service threads 2023-08-15 15:59:40,407 INFO [M:0;jenkins-hbase4:32961] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-15 15:59:40,408 ERROR [M:0;jenkins-hbase4:32961] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-15 15:59:40,408 INFO [M:0;jenkins-hbase4:32961] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-15 15:59:40,408 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-15 15:59:40,408 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:40,408 DEBUG [M:0;jenkins-hbase4:32961] zookeeper.ZKUtil(398): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-15 15:59:40,408 WARN [M:0;jenkins-hbase4:32961] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-15 15:59:40,408 INFO [M:0;jenkins-hbase4:32961] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-15 15:59:40,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 15:59:40,409 INFO [M:0;jenkins-hbase4:32961] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-15 15:59:40,409 DEBUG [M:0;jenkins-hbase4:32961] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-15 15:59:40,409 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-15 15:59:40,409 INFO [M:0;jenkins-hbase4:32961] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:59:40,410 DEBUG [M:0;jenkins-hbase4:32961] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:59:40,410 DEBUG [M:0;jenkins-hbase4:32961] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-15 15:59:40,410 DEBUG [M:0;jenkins-hbase4:32961] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:59:40,410 INFO [M:0;jenkins-hbase4:32961] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.89 KB heapSize=47.33 KB 2023-08-15 15:59:40,432 INFO [M:0;jenkins-hbase4:32961] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.89 KB at sequenceid=100 (bloomFilter=true), to=hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/63a1c3be803f4263b31d6d56de69ff89 2023-08-15 15:59:40,439 INFO [M:0;jenkins-hbase4:32961] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a1c3be803f4263b31d6d56de69ff89 2023-08-15 15:59:40,441 DEBUG [M:0;jenkins-hbase4:32961] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/63a1c3be803f4263b31d6d56de69ff89 as hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/63a1c3be803f4263b31d6d56de69ff89 2023-08-15 15:59:40,447 INFO [M:0;jenkins-hbase4:32961] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a1c3be803f4263b31d6d56de69ff89 2023-08-15 15:59:40,447 INFO [M:0;jenkins-hbase4:32961] regionserver.HStore(1080): Added hdfs://localhost:37617/user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/63a1c3be803f4263b31d6d56de69ff89, entries=11, sequenceid=100, filesize=6.1 K 2023-08-15 15:59:40,448 INFO [M:0;jenkins-hbase4:32961] regionserver.HRegion(2948): Finished flush of dataSize ~38.89 KB/39824, heapSize ~47.31 KB/48448, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 38ms, sequenceid=100, compaction requested=false 2023-08-15 15:59:40,454 INFO [M:0;jenkins-hbase4:32961] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:59:40,454 DEBUG [M:0;jenkins-hbase4:32961] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 15:59:40,458 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/5e9d3c7a-1231-585a-677a-0b700b14611d/MasterData/WALs/jenkins-hbase4.apache.org,32961,1692115118452/jenkins-hbase4.apache.org%2C32961%2C1692115118452.1692115118589 not finished, retry = 0 2023-08-15 15:59:40,503 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 15:59:40,503 INFO [RS:0;jenkins-hbase4:38177] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38177,1692115118493; zookeeper connection closed. 2023-08-15 15:59:40,503 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): regionserver:38177-0x10010aae1070001, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 15:59:40,505 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@282ed44c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@282ed44c 2023-08-15 15:59:40,506 INFO [Listener at localhost/40391] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-15 15:59:40,559 INFO [M:0;jenkins-hbase4:32961] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-15 15:59:40,559 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-15 15:59:40,560 INFO [M:0;jenkins-hbase4:32961] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:32961 2023-08-15 15:59:40,562 DEBUG [M:0;jenkins-hbase4:32961] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,32961,1692115118452 already deleted, retry=false 2023-08-15 15:59:40,664 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 15:59:40,664 INFO [M:0;jenkins-hbase4:32961] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,32961,1692115118452; zookeeper connection closed. 2023-08-15 15:59:40,665 DEBUG [Listener at localhost/40391-EventThread] zookeeper.ZKWatcher(600): master:32961-0x10010aae1070000, quorum=127.0.0.1:64981, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 15:59:40,667 WARN [Listener at localhost/40391] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 15:59:40,679 INFO [Listener at localhost/40391] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:59:40,767 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-15 15:59:40,784 WARN [BP-1737645145-172.31.14.131-1692115117876 heartbeating to localhost/127.0.0.1:37617] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 15:59:40,784 WARN [BP-1737645145-172.31.14.131-1692115117876 heartbeating to localhost/127.0.0.1:37617] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1737645145-172.31.14.131-1692115117876 (Datanode Uuid 06ff890a-b1e5-432c-89fd-fd307d3eb110) service to localhost/127.0.0.1:37617 2023-08-15 15:59:40,785 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/cluster_b40e1168-a372-e19d-5164-059b57279616/dfs/data/data3/current/BP-1737645145-172.31.14.131-1692115117876] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:59:40,786 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/cluster_b40e1168-a372-e19d-5164-059b57279616/dfs/data/data4/current/BP-1737645145-172.31.14.131-1692115117876] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:59:40,787 WARN [Listener at localhost/40391] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 15:59:40,790 INFO [Listener at localhost/40391] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:59:40,894 WARN [BP-1737645145-172.31.14.131-1692115117876 heartbeating to localhost/127.0.0.1:37617] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 15:59:40,894 WARN [BP-1737645145-172.31.14.131-1692115117876 heartbeating to localhost/127.0.0.1:37617] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1737645145-172.31.14.131-1692115117876 (Datanode Uuid a0e70078-215c-4e63-97d4-a5783157dc13) service to localhost/127.0.0.1:37617 2023-08-15 15:59:40,895 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/cluster_b40e1168-a372-e19d-5164-059b57279616/dfs/data/data1/current/BP-1737645145-172.31.14.131-1692115117876] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:59:40,895 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/cluster_b40e1168-a372-e19d-5164-059b57279616/dfs/data/data2/current/BP-1737645145-172.31.14.131-1692115117876] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 15:59:40,909 INFO [Listener at localhost/40391] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 15:59:40,922 INFO [Listener at localhost/40391] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-15 15:59:40,948 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-15 15:59:40,958 INFO [Listener at localhost/40391] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=64 (was 78), OpenFileDescriptor=430 (was 467), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=91 (was 131), ProcessCount=169 (was 169), AvailableMemoryMB=685 (was 947) 2023-08-15 15:59:40,966 INFO [Listener at localhost/40391] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=65, OpenFileDescriptor=430, MaxFileDescriptor=60000, SystemLoadAverage=91, ProcessCount=169, AvailableMemoryMB=685 2023-08-15 15:59:40,966 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-15 15:59:40,966 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/hadoop.log.dir so I do NOT create it in target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a 2023-08-15 15:59:40,966 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9b19d913-af29-1187-a13e-98560bc02797/hadoop.tmp.dir so I do NOT create it in target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a 2023-08-15 15:59:40,967 INFO [Listener at localhost/40391] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/cluster_505db672-224e-5593-010e-aed0c4218503, deleteOnExit=true 2023-08-15 15:59:40,967 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-15 15:59:40,967 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/test.cache.data in system properties and HBase conf 2023-08-15 15:59:40,967 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/hadoop.tmp.dir in system properties and HBase conf 2023-08-15 15:59:40,967 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/hadoop.log.dir in system properties and HBase conf 2023-08-15 15:59:40,967 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-15 15:59:40,967 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-15 15:59:40,967 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-15 15:59:40,967 DEBUG [Listener at localhost/40391] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-15 15:59:40,968 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-15 15:59:40,969 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/nfs.dump.dir in system properties and HBase conf 2023-08-15 15:59:40,969 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/java.io.tmpdir in system properties and HBase conf 2023-08-15 15:59:40,969 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-15 15:59:40,969 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-15 15:59:40,969 INFO [Listener at localhost/40391] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-15 15:59:40,973 WARN [Listener at localhost/40391] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-15 15:59:40,973 WARN [Listener at localhost/40391] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-15 15:59:41,016 WARN [Listener at localhost/40391] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:59:41,019 INFO [Listener at localhost/40391] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:59:41,024 INFO [Listener at localhost/40391] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/java.io.tmpdir/Jetty_localhost_39501_hdfs____w74vay/webapp 2023-08-15 15:59:41,123 INFO [Listener at localhost/40391] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39501 2023-08-15 15:59:41,127 WARN [Listener at localhost/40391] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-15 15:59:41,128 WARN [Listener at localhost/40391] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-15 15:59:41,185 WARN [Listener at localhost/45523] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:59:41,206 WARN [Listener at localhost/45523] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:59:41,208 WARN [Listener at localhost/45523] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:59:41,209 INFO [Listener at localhost/45523] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:59:41,214 INFO [Listener at localhost/45523] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/java.io.tmpdir/Jetty_localhost_35873_datanode____3vtnix/webapp 2023-08-15 15:59:41,315 INFO [Listener at localhost/45523] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35873 2023-08-15 15:59:41,325 WARN [Listener at localhost/36583] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:59:41,346 WARN [Listener at localhost/36583] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 15:59:41,350 WARN [Listener at localhost/36583] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 15:59:41,352 INFO [Listener at localhost/36583] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 15:59:41,355 INFO [Listener at localhost/36583] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/java.io.tmpdir/Jetty_localhost_34481_datanode____.f0ymlp/webapp 2023-08-15 15:59:41,443 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3cf11b77116d7e95: Processing first storage report for DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0 from datanode addd2e78-719f-4683-ba13-a7a9b0d9e679 2023-08-15 15:59:41,443 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3cf11b77116d7e95: from storage DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0 node DatanodeRegistration(127.0.0.1:36497, datanodeUuid=addd2e78-719f-4683-ba13-a7a9b0d9e679, infoPort=43357, infoSecurePort=0, ipcPort=36583, storageInfo=lv=-57;cid=testClusterID;nsid=1492541730;c=1692115180976), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:59:41,443 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3cf11b77116d7e95: Processing first storage report for DS-64f5c21b-caf8-43af-8adf-c0a8281cdfb0 from datanode addd2e78-719f-4683-ba13-a7a9b0d9e679 2023-08-15 15:59:41,443 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3cf11b77116d7e95: from storage DS-64f5c21b-caf8-43af-8adf-c0a8281cdfb0 node DatanodeRegistration(127.0.0.1:36497, datanodeUuid=addd2e78-719f-4683-ba13-a7a9b0d9e679, infoPort=43357, infoSecurePort=0, ipcPort=36583, storageInfo=lv=-57;cid=testClusterID;nsid=1492541730;c=1692115180976), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:59:41,466 INFO [Listener at localhost/36583] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34481 2023-08-15 15:59:41,473 WARN [Listener at localhost/33883] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 15:59:41,646 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd6a9e070f4686f4c: Processing first storage report for DS-03d21dd4-76d0-4db8-9e77-533db29d9b76 from datanode fabdd565-92ff-4093-bd2c-eaa59afd4719 2023-08-15 15:59:41,646 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd6a9e070f4686f4c: from storage DS-03d21dd4-76d0-4db8-9e77-533db29d9b76 node DatanodeRegistration(127.0.0.1:42481, datanodeUuid=fabdd565-92ff-4093-bd2c-eaa59afd4719, infoPort=44839, infoSecurePort=0, ipcPort=33883, storageInfo=lv=-57;cid=testClusterID;nsid=1492541730;c=1692115180976), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:59:41,646 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd6a9e070f4686f4c: Processing first storage report for DS-23732d12-c9bd-4e31-8959-ecd4b9beec78 from datanode fabdd565-92ff-4093-bd2c-eaa59afd4719 2023-08-15 15:59:41,646 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd6a9e070f4686f4c: from storage DS-23732d12-c9bd-4e31-8959-ecd4b9beec78 node DatanodeRegistration(127.0.0.1:42481, datanodeUuid=fabdd565-92ff-4093-bd2c-eaa59afd4719, infoPort=44839, infoSecurePort=0, ipcPort=33883, storageInfo=lv=-57;cid=testClusterID;nsid=1492541730;c=1692115180976), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 15:59:41,682 DEBUG [Listener at localhost/33883] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a 2023-08-15 15:59:41,684 INFO [Listener at localhost/33883] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/cluster_505db672-224e-5593-010e-aed0c4218503/zookeeper_0, clientPort=50312, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/cluster_505db672-224e-5593-010e-aed0c4218503/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/cluster_505db672-224e-5593-010e-aed0c4218503/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-15 15:59:41,686 INFO [Listener at localhost/33883] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=50312 2023-08-15 15:59:41,686 INFO [Listener at localhost/33883] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:59:41,687 INFO [Listener at localhost/33883] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:59:41,707 INFO [Listener at localhost/33883] util.FSUtils(471): Created version file at hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e with version=8 2023-08-15 15:59:41,708 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/hbase-staging 2023-08-15 15:59:41,710 INFO [Listener at localhost/33883] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-15 15:59:41,710 INFO [Listener at localhost/33883] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:59:41,710 INFO [Listener at localhost/33883] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-15 15:59:41,710 INFO [Listener at localhost/33883] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-15 15:59:41,710 INFO [Listener at localhost/33883] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:59:41,710 INFO [Listener at localhost/33883] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-15 15:59:41,710 INFO [Listener at localhost/33883] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-15 15:59:41,712 INFO [Listener at localhost/33883] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34795 2023-08-15 15:59:41,712 INFO [Listener at localhost/33883] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:59:41,713 INFO [Listener at localhost/33883] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:59:41,714 INFO [Listener at localhost/33883] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34795 connecting to ZooKeeper ensemble=127.0.0.1:50312 2023-08-15 15:59:41,722 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:347950x0, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-15 15:59:41,723 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34795-0x10010abd81a0000 connected 2023-08-15 15:59:41,742 DEBUG [Listener at localhost/33883] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 15:59:41,742 DEBUG [Listener at localhost/33883] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:59:41,743 DEBUG [Listener at localhost/33883] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-15 15:59:41,743 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34795 2023-08-15 15:59:41,745 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34795 2023-08-15 15:59:41,745 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34795 2023-08-15 15:59:41,746 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34795 2023-08-15 15:59:41,746 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34795 2023-08-15 15:59:41,746 INFO [Listener at localhost/33883] master.HMaster(444): hbase.rootdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e, hbase.cluster.distributed=false 2023-08-15 15:59:41,761 INFO [Listener at localhost/33883] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-15 15:59:41,761 INFO [Listener at localhost/33883] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:59:41,761 INFO [Listener at localhost/33883] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-15 15:59:41,761 INFO [Listener at localhost/33883] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-15 15:59:41,761 INFO [Listener at localhost/33883] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 15:59:41,761 INFO [Listener at localhost/33883] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-15 15:59:41,761 INFO [Listener at localhost/33883] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-15 15:59:41,762 INFO [Listener at localhost/33883] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36385 2023-08-15 15:59:41,763 INFO [Listener at localhost/33883] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-15 15:59:41,764 DEBUG [Listener at localhost/33883] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-15 15:59:41,764 INFO [Listener at localhost/33883] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:59:41,765 INFO [Listener at localhost/33883] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:59:41,766 INFO [Listener at localhost/33883] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36385 connecting to ZooKeeper ensemble=127.0.0.1:50312 2023-08-15 15:59:41,769 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): regionserver:363850x0, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-15 15:59:41,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36385-0x10010abd81a0001 connected 2023-08-15 15:59:41,771 DEBUG [Listener at localhost/33883] zookeeper.ZKUtil(164): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 15:59:41,771 DEBUG [Listener at localhost/33883] zookeeper.ZKUtil(164): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 15:59:41,772 DEBUG [Listener at localhost/33883] zookeeper.ZKUtil(164): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-15 15:59:41,772 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36385 2023-08-15 15:59:41,772 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36385 2023-08-15 15:59:41,773 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36385 2023-08-15 15:59:41,775 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36385 2023-08-15 15:59:41,775 DEBUG [Listener at localhost/33883] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36385 2023-08-15 15:59:41,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 15:59:41,778 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-15 15:59:41,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 15:59:41,780 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-15 15:59:41,780 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-15 15:59:41,780 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:41,781 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-15 15:59:41,781 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,34795,1692115181709 from backup master directory 2023-08-15 15:59:41,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-15 15:59:41,788 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 15:59:41,788 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-15 15:59:41,788 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-15 15:59:41,788 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 15:59:41,812 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/hbase.id with ID: 55854df2-794c-43c1-a009-40b9f5f19257 2023-08-15 15:59:41,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:59:41,834 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:42,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4a14185f to 127.0.0.1:50312 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:59:42,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6223816a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:59:42,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-15 15:59:42,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-15 15:59:42,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:59:42,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store-tmp 2023-08-15 15:59:42,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:59:42,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-15 15:59:42,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:59:42,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:59:42,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-15 15:59:42,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:59:42,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 15:59:42,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 15:59:42,276 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/WALs/jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 15:59:42,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34795%2C1692115181709, suffix=, logDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/WALs/jenkins-hbase4.apache.org,34795,1692115181709, archiveDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/oldWALs, maxLogs=10 2023-08-15 15:59:42,294 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK] 2023-08-15 15:59:42,294 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK] 2023-08-15 15:59:42,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/WALs/jenkins-hbase4.apache.org,34795,1692115181709/jenkins-hbase4.apache.org%2C34795%2C1692115181709.1692115182280 2023-08-15 15:59:42,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK], DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK]] 2023-08-15 15:59:42,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:59:42,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:59:42,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:59:42,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:59:42,299 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:59:42,301 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-15 15:59:42,301 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-15 15:59:42,302 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:42,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:59:42,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:59:42,306 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 15:59:42,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:59:42,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=841147, jitterRate=0.06957407295703888}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:59:42,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 15:59:42,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-15 15:59:42,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-15 15:59:42,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-15 15:59:42,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-15 15:59:42,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-15 15:59:42,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-15 15:59:42,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-15 15:59:42,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-15 15:59:42,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-15 15:59:42,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-15 15:59:42,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-15 15:59:42,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-15 15:59:42,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-15 15:59:42,324 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-15 15:59:42,326 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:42,326 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-15 15:59:42,327 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-15 15:59:42,327 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-15 15:59:42,330 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-15 15:59:42,330 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-15 15:59:42,330 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:42,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,34795,1692115181709, sessionid=0x10010abd81a0000, setting cluster-up flag (Was=false) 2023-08-15 15:59:42,333 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:42,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-15 15:59:42,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 15:59:42,340 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:42,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-15 15:59:42,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 15:59:42,346 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.hbase-snapshot/.tmp 2023-08-15 15:59:42,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-15 15:59:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:59:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:59:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:59:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 15:59:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-15 15:59:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-15 15:59:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692115212354 2023-08-15 15:59:42,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-15 15:59:42,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-15 15:59:42,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-15 15:59:42,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-15 15:59:42,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-15 15:59:42,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-15 15:59:42,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,355 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-15 15:59:42,355 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-15 15:59:42,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-15 15:59:42,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-15 15:59:42,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-15 15:59:42,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-15 15:59:42,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-15 15:59:42,356 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-15 15:59:42,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115182356,5,FailOnTimeoutGroup] 2023-08-15 15:59:42,357 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115182357,5,FailOnTimeoutGroup] 2023-08-15 15:59:42,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-15 15:59:42,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,367 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-15 15:59:42,368 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-15 15:59:42,368 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e 2023-08-15 15:59:42,377 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(951): ClusterId : 55854df2-794c-43c1-a009-40b9f5f19257 2023-08-15 15:59:42,378 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:59:42,378 DEBUG [RS:0;jenkins-hbase4:36385] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-15 15:59:42,380 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-15 15:59:42,380 DEBUG [RS:0;jenkins-hbase4:36385] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-15 15:59:42,380 DEBUG [RS:0;jenkins-hbase4:36385] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-15 15:59:42,381 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/info 2023-08-15 15:59:42,382 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-15 15:59:42,382 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:42,382 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-15 15:59:42,383 DEBUG [RS:0;jenkins-hbase4:36385] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-15 15:59:42,384 DEBUG [RS:0;jenkins-hbase4:36385] zookeeper.ReadOnlyZKClient(139): Connect 0x0f7724a4 to 127.0.0.1:50312 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:59:42,384 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:59:42,385 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-15 15:59:42,386 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:42,386 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-15 15:59:42,388 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/table 2023-08-15 15:59:42,388 DEBUG [RS:0;jenkins-hbase4:36385] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b0db3a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:59:42,388 DEBUG [RS:0;jenkins-hbase4:36385] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@58f4fd4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-15 15:59:42,388 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-15 15:59:42,389 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:42,390 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740 2023-08-15 15:59:42,391 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740 2023-08-15 15:59:42,394 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-15 15:59:42,395 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-15 15:59:42,400 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:59:42,400 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=788748, jitterRate=0.002945840358734131}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-15 15:59:42,400 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-15 15:59:42,400 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-15 15:59:42,400 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-15 15:59:42,400 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-15 15:59:42,400 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-15 15:59:42,401 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-15 15:59:42,401 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36385 2023-08-15 15:59:42,401 INFO [RS:0;jenkins-hbase4:36385] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-15 15:59:42,401 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-15 15:59:42,401 INFO [RS:0;jenkins-hbase4:36385] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-15 15:59:42,401 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-15 15:59:42,401 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1022): About to register with Master. 2023-08-15 15:59:42,402 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,34795,1692115181709 with isa=jenkins-hbase4.apache.org/172.31.14.131:36385, startcode=1692115181760 2023-08-15 15:59:42,402 DEBUG [RS:0;jenkins-hbase4:36385] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-15 15:59:42,402 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-15 15:59:42,402 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-15 15:59:42,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-15 15:59:42,404 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-15 15:59:42,405 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59781, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-15 15:59:42,406 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34795] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:42,406 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-15 15:59:42,406 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e 2023-08-15 15:59:42,407 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45523 2023-08-15 15:59:42,407 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-15 15:59:42,408 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 15:59:42,409 DEBUG [RS:0;jenkins-hbase4:36385] zookeeper.ZKUtil(162): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:42,409 WARN [RS:0;jenkins-hbase4:36385] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-15 15:59:42,409 INFO [RS:0;jenkins-hbase4:36385] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:59:42,409 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:42,414 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36385,1692115181760] 2023-08-15 15:59:42,416 DEBUG [RS:0;jenkins-hbase4:36385] zookeeper.ZKUtil(162): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:42,417 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-15 15:59:42,417 INFO [RS:0;jenkins-hbase4:36385] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-15 15:59:42,419 INFO [RS:0;jenkins-hbase4:36385] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-15 15:59:42,419 INFO [RS:0;jenkins-hbase4:36385] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-15 15:59:42,419 INFO [RS:0;jenkins-hbase4:36385] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,419 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-15 15:59:42,420 INFO [RS:0;jenkins-hbase4:36385] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,421 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,421 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,421 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,421 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,421 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,421 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-15 15:59:42,422 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,422 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,422 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,422 DEBUG [RS:0;jenkins-hbase4:36385] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 15:59:42,422 INFO [RS:0;jenkins-hbase4:36385] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,423 INFO [RS:0;jenkins-hbase4:36385] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,423 INFO [RS:0;jenkins-hbase4:36385] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,433 INFO [RS:0;jenkins-hbase4:36385] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-15 15:59:42,434 INFO [RS:0;jenkins-hbase4:36385] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36385,1692115181760-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,444 INFO [RS:0;jenkins-hbase4:36385] regionserver.Replication(203): jenkins-hbase4.apache.org,36385,1692115181760 started 2023-08-15 15:59:42,444 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36385,1692115181760, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36385, sessionid=0x10010abd81a0001 2023-08-15 15:59:42,444 DEBUG [RS:0;jenkins-hbase4:36385] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-15 15:59:42,444 DEBUG [RS:0;jenkins-hbase4:36385] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:42,444 DEBUG [RS:0;jenkins-hbase4:36385] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36385,1692115181760' 2023-08-15 15:59:42,444 DEBUG [RS:0;jenkins-hbase4:36385] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 15:59:42,445 DEBUG [RS:0;jenkins-hbase4:36385] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 15:59:42,445 DEBUG [RS:0;jenkins-hbase4:36385] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-15 15:59:42,445 DEBUG [RS:0;jenkins-hbase4:36385] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-15 15:59:42,445 DEBUG [RS:0;jenkins-hbase4:36385] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:42,445 DEBUG [RS:0;jenkins-hbase4:36385] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36385,1692115181760' 2023-08-15 15:59:42,445 DEBUG [RS:0;jenkins-hbase4:36385] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-15 15:59:42,446 DEBUG [RS:0;jenkins-hbase4:36385] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-15 15:59:42,446 DEBUG [RS:0;jenkins-hbase4:36385] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-15 15:59:42,446 INFO [RS:0;jenkins-hbase4:36385] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-15 15:59:42,446 INFO [RS:0;jenkins-hbase4:36385] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-15 15:59:42,548 INFO [RS:0;jenkins-hbase4:36385] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36385%2C1692115181760, suffix=, logDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760, archiveDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/oldWALs, maxLogs=32 2023-08-15 15:59:42,557 DEBUG [jenkins-hbase4:34795] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-15 15:59:42,559 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36385,1692115181760, state=OPENING 2023-08-15 15:59:42,561 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-15 15:59:42,563 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:42,564 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-15 15:59:42,564 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36385,1692115181760}] 2023-08-15 15:59:42,573 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK] 2023-08-15 15:59:42,573 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK] 2023-08-15 15:59:42,577 INFO [RS:0;jenkins-hbase4:36385] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115182549 2023-08-15 15:59:42,577 DEBUG [RS:0;jenkins-hbase4:36385] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK], DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK]] 2023-08-15 15:59:42,725 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:42,725 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-15 15:59:42,727 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49850, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-15 15:59:42,731 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-15 15:59:42,732 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 15:59:42,733 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36385%2C1692115181760.meta, suffix=.meta, logDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760, archiveDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/oldWALs, maxLogs=32 2023-08-15 15:59:42,748 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK] 2023-08-15 15:59:42,748 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK] 2023-08-15 15:59:42,750 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.meta.1692115182734.meta 2023-08-15 15:59:42,750 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK], DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK]] 2023-08-15 15:59:42,750 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:59:42,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-15 15:59:42,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-15 15:59:42,751 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-15 15:59:42,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-15 15:59:42,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:59:42,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-15 15:59:42,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-15 15:59:42,752 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-15 15:59:42,753 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/info 2023-08-15 15:59:42,753 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/info 2023-08-15 15:59:42,754 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-15 15:59:42,754 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:42,755 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-15 15:59:42,755 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:59:42,755 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/rep_barrier 2023-08-15 15:59:42,756 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-15 15:59:42,756 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:42,756 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-15 15:59:42,757 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/table 2023-08-15 15:59:42,757 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/table 2023-08-15 15:59:42,757 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-15 15:59:42,758 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:42,758 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740 2023-08-15 15:59:42,760 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740 2023-08-15 15:59:42,762 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-15 15:59:42,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-15 15:59:42,764 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=768151, jitterRate=-0.023246660828590393}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-15 15:59:42,764 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-15 15:59:42,765 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692115182725 2023-08-15 15:59:42,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-15 15:59:42,769 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-15 15:59:42,769 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36385,1692115181760, state=OPEN 2023-08-15 15:59:42,771 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-15 15:59:42,771 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-15 15:59:42,773 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-15 15:59:42,773 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36385,1692115181760 in 207 msec 2023-08-15 15:59:42,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-15 15:59:42,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 371 msec 2023-08-15 15:59:42,778 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 429 msec 2023-08-15 15:59:42,778 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692115182778, completionTime=-1 2023-08-15 15:59:42,778 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-15 15:59:42,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-15 15:59:42,780 DEBUG [hconnection-0x3468f11b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-15 15:59:42,782 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49858, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-15 15:59:42,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-15 15:59:42,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692115242783 2023-08-15 15:59:42,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692115302783 2023-08-15 15:59:42,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-15 15:59:42,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34795,1692115181709-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34795,1692115181709-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34795,1692115181709-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:34795, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-15 15:59:42,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-15 15:59:42,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-15 15:59:42,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-15 15:59:42,794 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-15 15:59:42,795 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-15 15:59:42,796 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-15 15:59:42,797 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:42,798 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87 empty. 2023-08-15 15:59:42,798 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:42,798 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-15 15:59:42,809 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-15 15:59:42,810 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 339df4b91c91c52e8aa5664386fa3b87, NAME => 'hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp 2023-08-15 15:59:42,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:59:42,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 339df4b91c91c52e8aa5664386fa3b87, disabling compactions & flushes 2023-08-15 15:59:42,821 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 15:59:42,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 15:59:42,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. after waiting 0 ms 2023-08-15 15:59:42,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 15:59:42,821 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 15:59:42,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 339df4b91c91c52e8aa5664386fa3b87: 2023-08-15 15:59:42,824 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-15 15:59:42,827 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115182826"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115182826"}]},"ts":"1692115182826"} 2023-08-15 15:59:42,829 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-15 15:59:42,830 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-15 15:59:42,830 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115182830"}]},"ts":"1692115182830"} 2023-08-15 15:59:42,832 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-15 15:59:42,840 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=339df4b91c91c52e8aa5664386fa3b87, ASSIGN}] 2023-08-15 15:59:42,843 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=339df4b91c91c52e8aa5664386fa3b87, ASSIGN 2023-08-15 15:59:42,847 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=339df4b91c91c52e8aa5664386fa3b87, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36385,1692115181760; forceNewPlan=false, retain=false 2023-08-15 15:59:42,998 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=339df4b91c91c52e8aa5664386fa3b87, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:42,999 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115182998"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115182998"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115182998"}]},"ts":"1692115182998"} 2023-08-15 15:59:43,002 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 339df4b91c91c52e8aa5664386fa3b87, server=jenkins-hbase4.apache.org,36385,1692115181760}] 2023-08-15 15:59:43,159 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 15:59:43,159 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 339df4b91c91c52e8aa5664386fa3b87, NAME => 'hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:59:43,159 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:43,159 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:59:43,159 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:43,159 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:43,161 INFO [StoreOpener-339df4b91c91c52e8aa5664386fa3b87-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:43,162 DEBUG [StoreOpener-339df4b91c91c52e8aa5664386fa3b87-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87/info 2023-08-15 15:59:43,162 DEBUG [StoreOpener-339df4b91c91c52e8aa5664386fa3b87-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87/info 2023-08-15 15:59:43,163 INFO [StoreOpener-339df4b91c91c52e8aa5664386fa3b87-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 339df4b91c91c52e8aa5664386fa3b87 columnFamilyName info 2023-08-15 15:59:43,163 INFO [StoreOpener-339df4b91c91c52e8aa5664386fa3b87-1] regionserver.HStore(310): Store=339df4b91c91c52e8aa5664386fa3b87/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:43,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:43,165 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:43,167 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 339df4b91c91c52e8aa5664386fa3b87 2023-08-15 15:59:43,169 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:59:43,170 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 339df4b91c91c52e8aa5664386fa3b87; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=732066, jitterRate=-0.06913074851036072}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:59:43,170 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 339df4b91c91c52e8aa5664386fa3b87: 2023-08-15 15:59:43,170 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87., pid=6, masterSystemTime=1692115183155 2023-08-15 15:59:43,173 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 15:59:43,173 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 15:59:43,174 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=339df4b91c91c52e8aa5664386fa3b87, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:43,174 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115183173"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115183173"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115183173"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115183173"}]},"ts":"1692115183173"} 2023-08-15 15:59:43,178 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-15 15:59:43,178 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 339df4b91c91c52e8aa5664386fa3b87, server=jenkins-hbase4.apache.org,36385,1692115181760 in 173 msec 2023-08-15 15:59:43,180 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-15 15:59:43,180 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=339df4b91c91c52e8aa5664386fa3b87, ASSIGN in 338 msec 2023-08-15 15:59:43,181 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-15 15:59:43,181 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115183181"}]},"ts":"1692115183181"} 2023-08-15 15:59:43,183 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-15 15:59:43,185 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-15 15:59:43,187 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 394 msec 2023-08-15 15:59:43,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-15 15:59:43,196 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:59:43,196 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:43,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-15 15:59:43,208 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:59:43,211 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 9 msec 2023-08-15 15:59:43,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-15 15:59:43,229 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-15 15:59:43,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 11 msec 2023-08-15 15:59:43,247 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-15 15:59:43,250 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-15 15:59:43,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.462sec 2023-08-15 15:59:43,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-15 15:59:43,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-15 15:59:43,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-15 15:59:43,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34795,1692115181709-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-15 15:59:43,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34795,1692115181709-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-15 15:59:43,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-15 15:59:43,278 DEBUG [Listener at localhost/33883] zookeeper.ReadOnlyZKClient(139): Connect 0x0733519d to 127.0.0.1:50312 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 15:59:43,282 DEBUG [Listener at localhost/33883] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65f9a60, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 15:59:43,283 DEBUG [hconnection-0x139452f1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-15 15:59:43,285 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49860, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-15 15:59:43,287 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 15:59:43,287 INFO [Listener at localhost/33883] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 15:59:43,290 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-15 15:59:43,290 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 15:59:43,291 INFO [Listener at localhost/33883] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-15 15:59:43,292 DEBUG [Listener at localhost/33883] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-15 15:59:43,295 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37956, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-15 15:59:43,296 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34795] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-08-15 15:59:43,296 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34795] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-08-15 15:59:43,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34795] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-15 15:59:43,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34795] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRolling 2023-08-15 15:59:43,301 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-08-15 15:59:43,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34795] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRolling" procId is: 9 2023-08-15 15:59:43,302 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-15 15:59:43,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34795] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-15 15:59:43,304 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,305 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473 empty. 2023-08-15 15:59:43,306 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,306 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRolling regions 2023-08-15 15:59:43,322 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp/data/default/TestLogRolling-testLogRolling/.tabledesc/.tableinfo.0000000001 2023-08-15 15:59:43,323 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5623bed68b8364b08ed7682e64962473, NAME => 'TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/.tmp 2023-08-15 15:59:43,335 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:59:43,335 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1604): Closing 5623bed68b8364b08ed7682e64962473, disabling compactions & flushes 2023-08-15 15:59:43,336 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 15:59:43,336 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 15:59:43,336 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. after waiting 0 ms 2023-08-15 15:59:43,336 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 15:59:43,336 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 15:59:43,336 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1558): Region close journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 15:59:43,339 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ADD_TO_META 2023-08-15 15:59:43,340 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692115183340"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115183340"}]},"ts":"1692115183340"} 2023-08-15 15:59:43,342 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-15 15:59:43,343 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-15 15:59:43,343 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115183343"}]},"ts":"1692115183343"} 2023-08-15 15:59:43,345 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLING in hbase:meta 2023-08-15 15:59:43,349 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=5623bed68b8364b08ed7682e64962473, ASSIGN}] 2023-08-15 15:59:43,351 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=5623bed68b8364b08ed7682e64962473, ASSIGN 2023-08-15 15:59:43,352 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=5623bed68b8364b08ed7682e64962473, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36385,1692115181760; forceNewPlan=false, retain=false 2023-08-15 15:59:43,503 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=5623bed68b8364b08ed7682e64962473, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:43,503 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692115183503"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115183503"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115183503"}]},"ts":"1692115183503"} 2023-08-15 15:59:43,505 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 5623bed68b8364b08ed7682e64962473, server=jenkins-hbase4.apache.org,36385,1692115181760}] 2023-08-15 15:59:43,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 15:59:43,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5623bed68b8364b08ed7682e64962473, NAME => 'TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.', STARTKEY => '', ENDKEY => ''} 2023-08-15 15:59:43,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 15:59:43,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,668 INFO [StoreOpener-5623bed68b8364b08ed7682e64962473-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,670 DEBUG [StoreOpener-5623bed68b8364b08ed7682e64962473-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info 2023-08-15 15:59:43,670 DEBUG [StoreOpener-5623bed68b8364b08ed7682e64962473-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info 2023-08-15 15:59:43,670 INFO [StoreOpener-5623bed68b8364b08ed7682e64962473-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5623bed68b8364b08ed7682e64962473 columnFamilyName info 2023-08-15 15:59:43,671 INFO [StoreOpener-5623bed68b8364b08ed7682e64962473-1] regionserver.HStore(310): Store=5623bed68b8364b08ed7682e64962473/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 15:59:43,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:43,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 15:59:43,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5623bed68b8364b08ed7682e64962473; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=848011, jitterRate=0.0783030092716217}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 15:59:43,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 15:59:43,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473., pid=11, masterSystemTime=1692115183658 2023-08-15 15:59:43,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 15:59:43,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 15:59:43,684 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=5623bed68b8364b08ed7682e64962473, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:43,684 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692115183684"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115183684"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115183684"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115183684"}]},"ts":"1692115183684"} 2023-08-15 15:59:43,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-15 15:59:43,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 5623bed68b8364b08ed7682e64962473, server=jenkins-hbase4.apache.org,36385,1692115181760 in 182 msec 2023-08-15 15:59:43,691 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-15 15:59:43,692 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=5623bed68b8364b08ed7682e64962473, ASSIGN in 340 msec 2023-08-15 15:59:43,693 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-15 15:59:43,693 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115183693"}]},"ts":"1692115183693"} 2023-08-15 15:59:43,695 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLED in hbase:meta 2023-08-15 15:59:43,698 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_POST_OPERATION 2023-08-15 15:59:43,700 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRolling in 402 msec 2023-08-15 15:59:45,714 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-15 15:59:48,417 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-15 15:59:48,418 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-15 15:59:48,419 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-08-15 15:59:53,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34795] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-08-15 15:59:53,304 INFO [Listener at localhost/33883] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRolling, procId: 9 completed 2023-08-15 15:59:53,307 DEBUG [Listener at localhost/33883] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRolling 2023-08-15 15:59:53,307 DEBUG [Listener at localhost/33883] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 15:59:53,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 5623bed68b8364b08ed7682e64962473 2023-08-15 15:59:53,325 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 5623bed68b8364b08ed7682e64962473 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-15 15:59:53,348 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=12 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/b70350cf370d4377bb23b1d2de713902 2023-08-15 15:59:53,362 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=5623bed68b8364b08ed7682e64962473, server=jenkins-hbase4.apache.org,36385,1692115181760 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-15 15:59:53,362 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/b70350cf370d4377bb23b1d2de713902 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b70350cf370d4377bb23b1d2de713902 2023-08-15 15:59:53,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] ipc.CallRunner(144): callId: 38 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:49860 deadline: 1692115203361, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=5623bed68b8364b08ed7682e64962473, server=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 15:59:53,369 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b70350cf370d4377bb23b1d2de713902, entries=8, sequenceid=12, filesize=13.0 K 2023-08-15 15:59:53,371 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 5623bed68b8364b08ed7682e64962473 in 46ms, sequenceid=12, compaction requested=false 2023-08-15 15:59:53,371 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 16:00:03,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:03,421 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 5623bed68b8364b08ed7682e64962473 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-15 16:00:03,441 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=37 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/53f2910aa8b14684a24437289c9d0641 2023-08-15 16:00:03,451 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/53f2910aa8b14684a24437289c9d0641 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/53f2910aa8b14684a24437289c9d0641 2023-08-15 16:00:03,458 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/53f2910aa8b14684a24437289c9d0641, entries=22, sequenceid=37, filesize=27.4 K 2023-08-15 16:00:03,459 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=2.05 KB/2102 for 5623bed68b8364b08ed7682e64962473 in 37ms, sequenceid=37, compaction requested=false 2023-08-15 16:00:03,459 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 16:00:03,459 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=40.3 K, sizeToCheck=16.0 K 2023-08-15 16:00:03,459 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-15 16:00:03,460 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/53f2910aa8b14684a24437289c9d0641 because midkey is the same as first or last row 2023-08-15 16:00:05,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:05,439 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 5623bed68b8364b08ed7682e64962473 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-15 16:00:05,457 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=48 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/0ad0a676c1d24016b3356f2513aa880b 2023-08-15 16:00:05,469 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/0ad0a676c1d24016b3356f2513aa880b as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/0ad0a676c1d24016b3356f2513aa880b 2023-08-15 16:00:05,484 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/0ad0a676c1d24016b3356f2513aa880b, entries=8, sequenceid=48, filesize=13.0 K 2023-08-15 16:00:05,486 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 5623bed68b8364b08ed7682e64962473 in 46ms, sequenceid=48, compaction requested=true 2023-08-15 16:00:05,486 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 16:00:05,486 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=53.3 K, sizeToCheck=16.0 K 2023-08-15 16:00:05,487 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-15 16:00:05,487 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/53f2910aa8b14684a24437289c9d0641 because midkey is the same as first or last row 2023-08-15 16:00:05,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:05,490 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:05,491 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 16:00:05,497 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 5623bed68b8364b08ed7682e64962473 1/1 column families, dataSize=23.61 KB heapSize=25.59 KB 2023-08-15 16:00:05,499 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 54576 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 16:00:05,499 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 5623bed68b8364b08ed7682e64962473/info is initiating minor compaction (all files) 2023-08-15 16:00:05,500 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5623bed68b8364b08ed7682e64962473/info in TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 16:00:05,500 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b70350cf370d4377bb23b1d2de713902, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/53f2910aa8b14684a24437289c9d0641, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/0ad0a676c1d24016b3356f2513aa880b] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp, totalSize=53.3 K 2023-08-15 16:00:05,501 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting b70350cf370d4377bb23b1d2de713902, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1692115193311 2023-08-15 16:00:05,502 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 53f2910aa8b14684a24437289c9d0641, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=37, earliestPutTs=1692115193327 2023-08-15 16:00:05,503 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 0ad0a676c1d24016b3356f2513aa880b, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1692115203422 2023-08-15 16:00:05,557 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.63 KB at sequenceid=75 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/43c52a660c6e41a4b889bf1c710a0578 2023-08-15 16:00:05,567 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5623bed68b8364b08ed7682e64962473#info#compaction#19 average throughput is 19.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:00:05,572 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/43c52a660c6e41a4b889bf1c710a0578 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/43c52a660c6e41a4b889bf1c710a0578 2023-08-15 16:00:05,601 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/43c52a660c6e41a4b889bf1c710a0578, entries=24, sequenceid=75, filesize=29.4 K 2023-08-15 16:00:05,603 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~24.63 KB/25224, heapSize ~26.67 KB/27312, currentSize=2.05 KB/2102 for 5623bed68b8364b08ed7682e64962473 in 106ms, sequenceid=75, compaction requested=false 2023-08-15 16:00:05,603 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 16:00:05,603 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=82.7 K, sizeToCheck=16.0 K 2023-08-15 16:00:05,604 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-15 16:00:05,604 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/43c52a660c6e41a4b889bf1c710a0578 because midkey is the same as first or last row 2023-08-15 16:00:05,620 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/b4ba5f72616d4a139b3486e4dc0043f6 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b4ba5f72616d4a139b3486e4dc0043f6 2023-08-15 16:00:05,630 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 5623bed68b8364b08ed7682e64962473/info of 5623bed68b8364b08ed7682e64962473 into b4ba5f72616d4a139b3486e4dc0043f6(size=43.9 K), total size for store is 73.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:00:05,630 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 16:00:05,630 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473., storeName=5623bed68b8364b08ed7682e64962473/info, priority=13, startTime=1692115205489; duration=0sec 2023-08-15 16:00:05,631 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=73.4 K, sizeToCheck=16.0 K 2023-08-15 16:00:05,631 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-15 16:00:05,631 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b4ba5f72616d4a139b3486e4dc0043f6 because midkey is the same as first or last row 2023-08-15 16:00:05,631 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:07,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:07,518 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 5623bed68b8364b08ed7682e64962473 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-15 16:00:07,558 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=87 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/e9c0068cab3a45dfa17d5cb6edd872f5 2023-08-15 16:00:07,567 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/e9c0068cab3a45dfa17d5cb6edd872f5 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/e9c0068cab3a45dfa17d5cb6edd872f5 2023-08-15 16:00:07,568 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=5623bed68b8364b08ed7682e64962473, server=jenkins-hbase4.apache.org,36385,1692115181760 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-15 16:00:07,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] ipc.CallRunner(144): callId: 102 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:49860 deadline: 1692115217568, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=5623bed68b8364b08ed7682e64962473, server=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:07,578 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/e9c0068cab3a45dfa17d5cb6edd872f5, entries=8, sequenceid=87, filesize=13.0 K 2023-08-15 16:00:07,580 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 5623bed68b8364b08ed7682e64962473 in 62ms, sequenceid=87, compaction requested=true 2023-08-15 16:00:07,580 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 16:00:07,580 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=86.3 K, sizeToCheck=16.0 K 2023-08-15 16:00:07,580 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-15 16:00:07,580 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b4ba5f72616d4a139b3486e4dc0043f6 because midkey is the same as first or last row 2023-08-15 16:00:07,580 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:07,580 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 16:00:07,582 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 88399 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 16:00:07,582 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 5623bed68b8364b08ed7682e64962473/info is initiating minor compaction (all files) 2023-08-15 16:00:07,582 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5623bed68b8364b08ed7682e64962473/info in TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 16:00:07,582 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b4ba5f72616d4a139b3486e4dc0043f6, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/43c52a660c6e41a4b889bf1c710a0578, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/e9c0068cab3a45dfa17d5cb6edd872f5] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp, totalSize=86.3 K 2023-08-15 16:00:07,583 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting b4ba5f72616d4a139b3486e4dc0043f6, keycount=38, bloomtype=ROW, size=43.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1692115193311 2023-08-15 16:00:07,583 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 43c52a660c6e41a4b889bf1c710a0578, keycount=24, bloomtype=ROW, size=29.4 K, encoding=NONE, compression=NONE, seqNum=75, earliestPutTs=1692115205440 2023-08-15 16:00:07,584 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting e9c0068cab3a45dfa17d5cb6edd872f5, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=87, earliestPutTs=1692115205498 2023-08-15 16:00:07,601 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5623bed68b8364b08ed7682e64962473#info#compaction#21 average throughput is 14.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:00:07,642 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/fa2534aec0334c239e2a126c1a995931 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931 2023-08-15 16:00:07,654 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 5623bed68b8364b08ed7682e64962473/info of 5623bed68b8364b08ed7682e64962473 into fa2534aec0334c239e2a126c1a995931(size=77.1 K), total size for store is 77.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:00:07,654 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 16:00:07,654 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473., storeName=5623bed68b8364b08ed7682e64962473/info, priority=13, startTime=1692115207580; duration=0sec 2023-08-15 16:00:07,654 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=77.1 K, sizeToCheck=16.0 K 2023-08-15 16:00:07,654 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-08-15 16:00:07,656 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit(227): Splitting TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473., compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:07,656 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:07,657 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34795] assignment.AssignmentManager(1140): Split request from jenkins-hbase4.apache.org,36385,1692115181760, parent={ENCODED => 5623bed68b8364b08ed7682e64962473, NAME => 'TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.', STARTKEY => '', ENDKEY => ''} splitKey=row0064 2023-08-15 16:00:07,668 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:07,678 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34795] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=5623bed68b8364b08ed7682e64962473, daughterA=93bcc30e744bc8bbeac8634baeccb04b, daughterB=3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:07,679 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=5623bed68b8364b08ed7682e64962473, daughterA=93bcc30e744bc8bbeac8634baeccb04b, daughterB=3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:07,679 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=5623bed68b8364b08ed7682e64962473, daughterA=93bcc30e744bc8bbeac8634baeccb04b, daughterB=3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:07,679 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=5623bed68b8364b08ed7682e64962473, daughterA=93bcc30e744bc8bbeac8634baeccb04b, daughterB=3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:07,691 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=5623bed68b8364b08ed7682e64962473, UNASSIGN}] 2023-08-15 16:00:07,693 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=5623bed68b8364b08ed7682e64962473, UNASSIGN 2023-08-15 16:00:07,694 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=5623bed68b8364b08ed7682e64962473, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:07,695 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692115207694"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115207694"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115207694"}]},"ts":"1692115207694"} 2023-08-15 16:00:07,697 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 5623bed68b8364b08ed7682e64962473, server=jenkins-hbase4.apache.org,36385,1692115181760}] 2023-08-15 16:00:07,859 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:07,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5623bed68b8364b08ed7682e64962473, disabling compactions & flushes 2023-08-15 16:00:07,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 16:00:07,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 16:00:07,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. after waiting 0 ms 2023-08-15 16:00:07,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 16:00:07,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 5623bed68b8364b08ed7682e64962473 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-08-15 16:00:07,907 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=112 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/9cc92b136e2747ba824feaaa2a425b86 2023-08-15 16:00:07,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.tmp/info/9cc92b136e2747ba824feaaa2a425b86 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/9cc92b136e2747ba824feaaa2a425b86 2023-08-15 16:00:07,920 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/9cc92b136e2747ba824feaaa2a425b86, entries=21, sequenceid=112, filesize=26.3 K 2023-08-15 16:00:07,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=0 B/0 for 5623bed68b8364b08ed7682e64962473 in 61ms, sequenceid=112, compaction requested=false 2023-08-15 16:00:07,928 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b70350cf370d4377bb23b1d2de713902, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/53f2910aa8b14684a24437289c9d0641, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b4ba5f72616d4a139b3486e4dc0043f6, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/0ad0a676c1d24016b3356f2513aa880b, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/43c52a660c6e41a4b889bf1c710a0578, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/e9c0068cab3a45dfa17d5cb6edd872f5] to archive 2023-08-15 16:00:07,930 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-15 16:00:07,933 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b70350cf370d4377bb23b1d2de713902 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b70350cf370d4377bb23b1d2de713902 2023-08-15 16:00:07,937 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/53f2910aa8b14684a24437289c9d0641 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/53f2910aa8b14684a24437289c9d0641 2023-08-15 16:00:07,939 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b4ba5f72616d4a139b3486e4dc0043f6 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/b4ba5f72616d4a139b3486e4dc0043f6 2023-08-15 16:00:07,941 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/0ad0a676c1d24016b3356f2513aa880b to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/0ad0a676c1d24016b3356f2513aa880b 2023-08-15 16:00:07,943 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/43c52a660c6e41a4b889bf1c710a0578 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/43c52a660c6e41a4b889bf1c710a0578 2023-08-15 16:00:07,945 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/e9c0068cab3a45dfa17d5cb6edd872f5 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/e9c0068cab3a45dfa17d5cb6edd872f5 2023-08-15 16:00:07,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/recovered.edits/115.seqid, newMaxSeqId=115, maxSeqId=1 2023-08-15 16:00:07,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. 2023-08-15 16:00:07,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5623bed68b8364b08ed7682e64962473: 2023-08-15 16:00:07,957 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:07,958 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=5623bed68b8364b08ed7682e64962473, regionState=CLOSED 2023-08-15 16:00:07,958 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692115207958"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115207958"}]},"ts":"1692115207958"} 2023-08-15 16:00:07,962 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-15 16:00:07,962 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 5623bed68b8364b08ed7682e64962473, server=jenkins-hbase4.apache.org,36385,1692115181760 in 263 msec 2023-08-15 16:00:07,964 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-15 16:00:07,964 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=5623bed68b8364b08ed7682e64962473, UNASSIGN in 271 msec 2023-08-15 16:00:07,981 INFO [PEWorker-4] assignment.SplitTableRegionProcedure(694): pid=12 splitting 2 storefiles, region=5623bed68b8364b08ed7682e64962473, threads=2 2023-08-15 16:00:07,982 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/9cc92b136e2747ba824feaaa2a425b86 for region: 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:07,982 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931 for region: 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:08,003 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/9cc92b136e2747ba824feaaa2a425b86, top=true 2023-08-15 16:00:08,026 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/.splits/3068f0f945bfd5efbdf204bef7e4c6b0/info/TestLogRolling-testLogRolling=5623bed68b8364b08ed7682e64962473-9cc92b136e2747ba824feaaa2a425b86 for child: 3068f0f945bfd5efbdf204bef7e4c6b0, parent: 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:08,026 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/9cc92b136e2747ba824feaaa2a425b86 for region: 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:08,049 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931 for region: 5623bed68b8364b08ed7682e64962473 2023-08-15 16:00:08,049 DEBUG [PEWorker-4] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 5623bed68b8364b08ed7682e64962473 Daughter A: 1 storefiles, Daughter B: 2 storefiles. 2023-08-15 16:00:08,494 DEBUG [PEWorker-4] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/recovered.edits/115.seqid, newMaxSeqId=115, maxSeqId=-1 2023-08-15 16:00:08,497 DEBUG [PEWorker-4] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/recovered.edits/115.seqid, newMaxSeqId=115, maxSeqId=-1 2023-08-15 16:00:08,499 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1692115208499"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1692115208499"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1692115208499"}]},"ts":"1692115208499"} 2023-08-15 16:00:08,499 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692115208499"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115208499"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115208499"}]},"ts":"1692115208499"} 2023-08-15 16:00:08,499 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692115208499"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115208499"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115208499"}]},"ts":"1692115208499"} 2023-08-15 16:00:08,540 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36385] regionserver.HRegion(9158): Flush requested on 1588230740 2023-08-15 16:00:08,540 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-08-15 16:00:08,540 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.82 KB heapSize=8.36 KB 2023-08-15 16:00:08,549 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=93bcc30e744bc8bbeac8634baeccb04b, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3068f0f945bfd5efbdf204bef7e4c6b0, ASSIGN}] 2023-08-15 16:00:08,551 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3068f0f945bfd5efbdf204bef7e4c6b0, ASSIGN 2023-08-15 16:00:08,551 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=93bcc30e744bc8bbeac8634baeccb04b, ASSIGN 2023-08-15 16:00:08,552 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3068f0f945bfd5efbdf204bef7e4c6b0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36385,1692115181760; forceNewPlan=false, retain=false 2023-08-15 16:00:08,554 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=93bcc30e744bc8bbeac8634baeccb04b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36385,1692115181760; forceNewPlan=false, retain=false 2023-08-15 16:00:08,555 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.61 KB at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/.tmp/info/9880f87099d9429280f0b06b4d2aff96 2023-08-15 16:00:08,570 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=216 B at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/.tmp/table/0d845794037e47ac9c201c2b299d1cf5 2023-08-15 16:00:08,575 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/.tmp/info/9880f87099d9429280f0b06b4d2aff96 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/info/9880f87099d9429280f0b06b4d2aff96 2023-08-15 16:00:08,580 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/info/9880f87099d9429280f0b06b4d2aff96, entries=29, sequenceid=17, filesize=8.6 K 2023-08-15 16:00:08,582 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/.tmp/table/0d845794037e47ac9c201c2b299d1cf5 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/table/0d845794037e47ac9c201c2b299d1cf5 2023-08-15 16:00:08,588 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/table/0d845794037e47ac9c201c2b299d1cf5, entries=4, sequenceid=17, filesize=4.8 K 2023-08-15 16:00:08,589 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.82 KB/4934, heapSize ~8.08 KB/8272, currentSize=0 B/0 for 1588230740 in 49ms, sequenceid=17, compaction requested=false 2023-08-15 16:00:08,589 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-08-15 16:00:08,704 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=93bcc30e744bc8bbeac8634baeccb04b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:08,704 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=3068f0f945bfd5efbdf204bef7e4c6b0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:08,704 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692115208704"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115208704"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115208704"}]},"ts":"1692115208704"} 2023-08-15 16:00:08,704 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692115208704"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115208704"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115208704"}]},"ts":"1692115208704"} 2023-08-15 16:00:08,706 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 93bcc30e744bc8bbeac8634baeccb04b, server=jenkins-hbase4.apache.org,36385,1692115181760}] 2023-08-15 16:00:08,707 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 3068f0f945bfd5efbdf204bef7e4c6b0, server=jenkins-hbase4.apache.org,36385,1692115181760}] 2023-08-15 16:00:08,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:00:08,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3068f0f945bfd5efbdf204bef7e4c6b0, NAME => 'TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.', STARTKEY => 'row0064', ENDKEY => ''} 2023-08-15 16:00:08,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:08,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 16:00:08,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:08,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:08,863 INFO [StoreOpener-3068f0f945bfd5efbdf204bef7e4c6b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:08,864 DEBUG [StoreOpener-3068f0f945bfd5efbdf204bef7e4c6b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info 2023-08-15 16:00:08,864 DEBUG [StoreOpener-3068f0f945bfd5efbdf204bef7e4c6b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info 2023-08-15 16:00:08,865 INFO [StoreOpener-3068f0f945bfd5efbdf204bef7e4c6b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3068f0f945bfd5efbdf204bef7e4c6b0 columnFamilyName info 2023-08-15 16:00:08,874 DEBUG [StoreOpener-3068f0f945bfd5efbdf204bef7e4c6b0-1] regionserver.HStore(539): loaded hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/TestLogRolling-testLogRolling=5623bed68b8364b08ed7682e64962473-9cc92b136e2747ba824feaaa2a425b86 2023-08-15 16:00:08,881 DEBUG [StoreOpener-3068f0f945bfd5efbdf204bef7e4c6b0-1] regionserver.HStore(539): loaded hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473->hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931-top 2023-08-15 16:00:08,881 INFO [StoreOpener-3068f0f945bfd5efbdf204bef7e4c6b0-1] regionserver.HStore(310): Store=3068f0f945bfd5efbdf204bef7e4c6b0/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:00:08,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:08,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:08,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:08,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3068f0f945bfd5efbdf204bef7e4c6b0; next sequenceid=116; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=763205, jitterRate=-0.02953489124774933}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 16:00:08,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:00:08,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0., pid=18, masterSystemTime=1692115208858 2023-08-15 16:00:08,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:08,889 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-08-15 16:00:08,890 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:00:08,890 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 3068f0f945bfd5efbdf204bef7e4c6b0/info is initiating minor compaction (all files) 2023-08-15 16:00:08,890 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3068f0f945bfd5efbdf204bef7e4c6b0/info in TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:00:08,891 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473->hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931-top, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/TestLogRolling-testLogRolling=5623bed68b8364b08ed7682e64962473-9cc92b136e2747ba824feaaa2a425b86] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp, totalSize=103.4 K 2023-08-15 16:00:08,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:00:08,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:00:08,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:00:08,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 93bcc30e744bc8bbeac8634baeccb04b, NAME => 'TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b.', STARTKEY => '', ENDKEY => 'row0064'} 2023-08-15 16:00:08,891 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473, keycount=35, bloomtype=ROW, size=77.1 K, encoding=NONE, compression=NONE, seqNum=88, earliestPutTs=1692115193311 2023-08-15 16:00:08,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:00:08,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 16:00:08,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:00:08,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:00:08,892 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=5623bed68b8364b08ed7682e64962473-9cc92b136e2747ba824feaaa2a425b86, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=112, earliestPutTs=1692115207519 2023-08-15 16:00:08,892 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=3068f0f945bfd5efbdf204bef7e4c6b0, regionState=OPEN, openSeqNum=116, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:08,893 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692115208892"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115208892"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115208892"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115208892"}]},"ts":"1692115208892"} 2023-08-15 16:00:08,893 INFO [StoreOpener-93bcc30e744bc8bbeac8634baeccb04b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:00:08,895 DEBUG [StoreOpener-93bcc30e744bc8bbeac8634baeccb04b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/info 2023-08-15 16:00:08,895 DEBUG [StoreOpener-93bcc30e744bc8bbeac8634baeccb04b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/info 2023-08-15 16:00:08,895 INFO [StoreOpener-93bcc30e744bc8bbeac8634baeccb04b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 93bcc30e744bc8bbeac8634baeccb04b columnFamilyName info 2023-08-15 16:00:08,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-08-15 16:00:08,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 3068f0f945bfd5efbdf204bef7e4c6b0, server=jenkins-hbase4.apache.org,36385,1692115181760 in 187 msec 2023-08-15 16:00:08,899 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=3068f0f945bfd5efbdf204bef7e4c6b0, ASSIGN in 349 msec 2023-08-15 16:00:08,903 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3068f0f945bfd5efbdf204bef7e4c6b0#info#compaction#25 average throughput is 28.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:00:08,913 DEBUG [StoreOpener-93bcc30e744bc8bbeac8634baeccb04b-1] regionserver.HStore(539): loaded hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473->hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931-bottom 2023-08-15 16:00:08,913 INFO [StoreOpener-93bcc30e744bc8bbeac8634baeccb04b-1] regionserver.HStore(310): Store=93bcc30e744bc8bbeac8634baeccb04b/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:00:08,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:00:08,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:00:08,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:00:08,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 93bcc30e744bc8bbeac8634baeccb04b; next sequenceid=116; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=715064, jitterRate=-0.09074969589710236}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 16:00:08,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 93bcc30e744bc8bbeac8634baeccb04b: 2023-08-15 16:00:08,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b., pid=17, masterSystemTime=1692115208858 2023-08-15 16:00:08,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:08,931 DEBUG [RS:0;jenkins-hbase4:36385-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-15 16:00:08,931 INFO [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:00:08,931 DEBUG [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.HStore(1912): 93bcc30e744bc8bbeac8634baeccb04b/info is initiating minor compaction (all files) 2023-08-15 16:00:08,931 INFO [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 93bcc30e744bc8bbeac8634baeccb04b/info in TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:00:08,932 INFO [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473->hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931-bottom] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/.tmp, totalSize=77.1 K 2023-08-15 16:00:08,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:00:08,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:00:08,934 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=93bcc30e744bc8bbeac8634baeccb04b, regionState=OPEN, openSeqNum=116, regionLocation=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:08,934 DEBUG [RS:0;jenkins-hbase4:36385-longCompactions-0] compactions.Compactor(207): Compacting fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473, keycount=35, bloomtype=ROW, size=77.1 K, encoding=NONE, compression=NONE, seqNum=87, earliestPutTs=1692115193311 2023-08-15 16:00:08,934 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692115208933"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115208933"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115208933"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115208933"}]},"ts":"1692115208933"} 2023-08-15 16:00:08,936 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/9681ecf3fa4347aaae55823c98318e5b as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/9681ecf3fa4347aaae55823c98318e5b 2023-08-15 16:00:08,939 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-08-15 16:00:08,940 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 93bcc30e744bc8bbeac8634baeccb04b, server=jenkins-hbase4.apache.org,36385,1692115181760 in 230 msec 2023-08-15 16:00:08,944 INFO [RS:0;jenkins-hbase4:36385-longCompactions-0] throttle.PressureAwareThroughputController(145): 93bcc30e744bc8bbeac8634baeccb04b#info#compaction#26 average throughput is 31.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:00:08,945 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-08-15 16:00:08,945 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=93bcc30e744bc8bbeac8634baeccb04b, ASSIGN in 391 msec 2023-08-15 16:00:08,947 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=5623bed68b8364b08ed7682e64962473, daughterA=93bcc30e744bc8bbeac8634baeccb04b, daughterB=3068f0f945bfd5efbdf204bef7e4c6b0 in 1.2770 sec 2023-08-15 16:00:08,951 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 (all) file(s) in 3068f0f945bfd5efbdf204bef7e4c6b0/info of 3068f0f945bfd5efbdf204bef7e4c6b0 into 9681ecf3fa4347aaae55823c98318e5b(size=33.7 K), total size for store is 33.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:00:08,951 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:00:08,951 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0., storeName=3068f0f945bfd5efbdf204bef7e4c6b0/info, priority=14, startTime=1692115208889; duration=0sec 2023-08-15 16:00:08,951 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:08,967 DEBUG [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/.tmp/info/8fa875856ba04d11a4e46f2b7ba008e3 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/info/8fa875856ba04d11a4e46f2b7ba008e3 2023-08-15 16:00:08,974 INFO [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.HStore(1652): Completed compaction of 1 (all) file(s) in 93bcc30e744bc8bbeac8634baeccb04b/info of 93bcc30e744bc8bbeac8634baeccb04b into 8fa875856ba04d11a4e46f2b7ba008e3(size=69.7 K), total size for store is 69.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:00:08,974 DEBUG [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 93bcc30e744bc8bbeac8634baeccb04b: 2023-08-15 16:00:08,974 INFO [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b., storeName=93bcc30e744bc8bbeac8634baeccb04b/info, priority=15, startTime=1692115208930; duration=0sec 2023-08-15 16:00:08,974 DEBUG [RS:0;jenkins-hbase4:36385-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:13,504 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-08-15 16:00:17,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] ipc.CallRunner(144): callId: 104 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:49860 deadline: 1692115227630, exception=org.apache.hadoop.hbase.NotServingRegionException: TestLogRolling-testLogRolling,,1692115183296.5623bed68b8364b08ed7682e64962473. is not online on jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:39,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:39,794 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-15 16:00:39,815 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=127 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/93c4ea9164d0454d82e06c62ffd2ec12 2023-08-15 16:00:39,822 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/93c4ea9164d0454d82e06c62ffd2ec12 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/93c4ea9164d0454d82e06c62ffd2ec12 2023-08-15 16:00:39,828 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/93c4ea9164d0454d82e06c62ffd2ec12, entries=8, sequenceid=127, filesize=13.0 K 2023-08-15 16:00:39,830 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=17.45 KB/17867 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 36ms, sequenceid=127, compaction requested=false 2023-08-15 16:00:39,830 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:00:39,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:39,831 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=18.47 KB heapSize=20.08 KB 2023-08-15 16:00:39,853 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=18.47 KB at sequenceid=148 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/dc642efc184542f39b23617899cfd419 2023-08-15 16:00:39,860 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/dc642efc184542f39b23617899cfd419 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/dc642efc184542f39b23617899cfd419 2023-08-15 16:00:39,866 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/dc642efc184542f39b23617899cfd419, entries=18, sequenceid=148, filesize=23.3 K 2023-08-15 16:00:39,867 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~18.47 KB/18918, heapSize ~20.06 KB/20544, currentSize=11.29 KB/11561 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 36ms, sequenceid=148, compaction requested=true 2023-08-15 16:00:39,867 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:00:39,867 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:39,867 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 16:00:39,868 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 71644 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 16:00:39,868 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 3068f0f945bfd5efbdf204bef7e4c6b0/info is initiating minor compaction (all files) 2023-08-15 16:00:39,868 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3068f0f945bfd5efbdf204bef7e4c6b0/info in TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:00:39,868 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/9681ecf3fa4347aaae55823c98318e5b, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/93c4ea9164d0454d82e06c62ffd2ec12, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/dc642efc184542f39b23617899cfd419] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp, totalSize=70.0 K 2023-08-15 16:00:39,869 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 9681ecf3fa4347aaae55823c98318e5b, keycount=28, bloomtype=ROW, size=33.7 K, encoding=NONE, compression=NONE, seqNum=112, earliestPutTs=1692115205500 2023-08-15 16:00:39,869 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 93c4ea9164d0454d82e06c62ffd2ec12, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=127, earliestPutTs=1692115237783 2023-08-15 16:00:39,870 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting dc642efc184542f39b23617899cfd419, keycount=18, bloomtype=ROW, size=23.3 K, encoding=NONE, compression=NONE, seqNum=148, earliestPutTs=1692115239795 2023-08-15 16:00:39,881 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3068f0f945bfd5efbdf204bef7e4c6b0#info#compaction#29 average throughput is 27.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:00:39,904 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/0f97807e8ea94719b99022c70c92a921 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0f97807e8ea94719b99022c70c92a921 2023-08-15 16:00:39,913 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3068f0f945bfd5efbdf204bef7e4c6b0/info of 3068f0f945bfd5efbdf204bef7e4c6b0 into 0f97807e8ea94719b99022c70c92a921(size=60.6 K), total size for store is 60.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:00:39,913 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:00:39,913 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0., storeName=3068f0f945bfd5efbdf204bef7e4c6b0/info, priority=13, startTime=1692115239867; duration=0sec 2023-08-15 16:00:39,914 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:41,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:41,856 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=12.32 KB heapSize=13.47 KB 2023-08-15 16:00:41,868 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=12.32 KB at sequenceid=164 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/1d2a51ccf3e6410591fc319b71c4bb5e 2023-08-15 16:00:41,875 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/1d2a51ccf3e6410591fc319b71c4bb5e as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/1d2a51ccf3e6410591fc319b71c4bb5e 2023-08-15 16:00:41,879 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=3068f0f945bfd5efbdf204bef7e4c6b0, server=jenkins-hbase4.apache.org,36385,1692115181760 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-15 16:00:41,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] ipc.CallRunner(144): callId: 163 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:49860 deadline: 1692115251879, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=3068f0f945bfd5efbdf204bef7e4c6b0, server=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:41,881 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/1d2a51ccf3e6410591fc319b71c4bb5e, entries=12, sequenceid=164, filesize=17.1 K 2023-08-15 16:00:41,882 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~12.32 KB/12612, heapSize ~13.45 KB/13776, currentSize=17.45 KB/17867 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 25ms, sequenceid=164, compaction requested=false 2023-08-15 16:00:41,882 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:00:51,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:00:51,953 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=18.47 KB heapSize=20.08 KB 2023-08-15 16:00:51,965 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=18.47 KB at sequenceid=185 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/b135acdf33c6457683f140683712ed83 2023-08-15 16:00:51,968 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=3068f0f945bfd5efbdf204bef7e4c6b0, server=jenkins-hbase4.apache.org,36385,1692115181760 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-15 16:00:51,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] ipc.CallRunner(144): callId: 177 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:49860 deadline: 1692115261968, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=3068f0f945bfd5efbdf204bef7e4c6b0, server=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:00:51,971 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/b135acdf33c6457683f140683712ed83 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/b135acdf33c6457683f140683712ed83 2023-08-15 16:00:51,977 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/b135acdf33c6457683f140683712ed83, entries=18, sequenceid=185, filesize=23.3 K 2023-08-15 16:00:51,977 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~18.47 KB/18918, heapSize ~20.06 KB/20544, currentSize=11.29 KB/11561 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 24ms, sequenceid=185, compaction requested=true 2023-08-15 16:00:51,977 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:00:51,978 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:00:51,978 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 16:00:51,979 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 103408 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 16:00:51,979 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 3068f0f945bfd5efbdf204bef7e4c6b0/info is initiating minor compaction (all files) 2023-08-15 16:00:51,979 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3068f0f945bfd5efbdf204bef7e4c6b0/info in TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:00:51,979 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0f97807e8ea94719b99022c70c92a921, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/1d2a51ccf3e6410591fc319b71c4bb5e, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/b135acdf33c6457683f140683712ed83] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp, totalSize=101.0 K 2023-08-15 16:00:51,979 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 0f97807e8ea94719b99022c70c92a921, keycount=54, bloomtype=ROW, size=60.6 K, encoding=NONE, compression=NONE, seqNum=148, earliestPutTs=1692115205500 2023-08-15 16:00:51,980 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 1d2a51ccf3e6410591fc319b71c4bb5e, keycount=12, bloomtype=ROW, size=17.1 K, encoding=NONE, compression=NONE, seqNum=164, earliestPutTs=1692115239831 2023-08-15 16:00:51,980 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting b135acdf33c6457683f140683712ed83, keycount=18, bloomtype=ROW, size=23.3 K, encoding=NONE, compression=NONE, seqNum=185, earliestPutTs=1692115241857 2023-08-15 16:00:51,992 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3068f0f945bfd5efbdf204bef7e4c6b0#info#compaction#32 average throughput is 28.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:00:52,007 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/8dbbcd97a6ed47b2b880210901fddbf8 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/8dbbcd97a6ed47b2b880210901fddbf8 2023-08-15 16:00:52,013 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3068f0f945bfd5efbdf204bef7e4c6b0/info of 3068f0f945bfd5efbdf204bef7e4c6b0 into 8dbbcd97a6ed47b2b880210901fddbf8(size=91.6 K), total size for store is 91.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:00:52,013 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:00:52,013 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0., storeName=3068f0f945bfd5efbdf204bef7e4c6b0/info, priority=13, startTime=1692115251977; duration=0sec 2023-08-15 16:00:52,013 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:02,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:02,070 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=12.32 KB heapSize=13.47 KB 2023-08-15 16:01:02,084 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=12.32 KB at sequenceid=201 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/7e5a3294b5a949e59c9eaf55ba821148 2023-08-15 16:01:02,090 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/7e5a3294b5a949e59c9eaf55ba821148 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7e5a3294b5a949e59c9eaf55ba821148 2023-08-15 16:01:02,096 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7e5a3294b5a949e59c9eaf55ba821148, entries=12, sequenceid=201, filesize=17.1 K 2023-08-15 16:01:02,096 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~12.32 KB/12612, heapSize ~13.45 KB/13776, currentSize=1.03 KB/1051 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 26ms, sequenceid=201, compaction requested=false 2023-08-15 16:01:02,097 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:04,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:04,080 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-15 16:01:04,092 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=212 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/4c4c24671f0b44319904c9367262c0ac 2023-08-15 16:01:04,099 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/4c4c24671f0b44319904c9367262c0ac as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4c4c24671f0b44319904c9367262c0ac 2023-08-15 16:01:04,105 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4c4c24671f0b44319904c9367262c0ac, entries=8, sequenceid=212, filesize=13.0 K 2023-08-15 16:01:04,106 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=16.42 KB/16816 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 26ms, sequenceid=212, compaction requested=true 2023-08-15 16:01:04,106 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:04,106 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:04,106 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 16:01:04,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:04,107 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=18.47 KB heapSize=20.08 KB 2023-08-15 16:01:04,107 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 124627 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 16:01:04,107 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 3068f0f945bfd5efbdf204bef7e4c6b0/info is initiating minor compaction (all files) 2023-08-15 16:01:04,108 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3068f0f945bfd5efbdf204bef7e4c6b0/info in TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:04,108 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/8dbbcd97a6ed47b2b880210901fddbf8, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7e5a3294b5a949e59c9eaf55ba821148, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4c4c24671f0b44319904c9367262c0ac] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp, totalSize=121.7 K 2023-08-15 16:01:04,108 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 8dbbcd97a6ed47b2b880210901fddbf8, keycount=84, bloomtype=ROW, size=91.6 K, encoding=NONE, compression=NONE, seqNum=185, earliestPutTs=1692115205500 2023-08-15 16:01:04,109 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 7e5a3294b5a949e59c9eaf55ba821148, keycount=12, bloomtype=ROW, size=17.1 K, encoding=NONE, compression=NONE, seqNum=201, earliestPutTs=1692115251954 2023-08-15 16:01:04,109 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 4c4c24671f0b44319904c9367262c0ac, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=212, earliestPutTs=1692115262071 2023-08-15 16:01:04,125 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=18.47 KB at sequenceid=233 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/0939472c58744132818f774ffb0d0b3a 2023-08-15 16:01:04,132 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3068f0f945bfd5efbdf204bef7e4c6b0#info#compaction#36 average throughput is 34.75 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:01:04,135 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/0939472c58744132818f774ffb0d0b3a as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0939472c58744132818f774ffb0d0b3a 2023-08-15 16:01:04,157 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0939472c58744132818f774ffb0d0b3a, entries=18, sequenceid=233, filesize=23.3 K 2023-08-15 16:01:04,158 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~18.47 KB/18918, heapSize ~20.06 KB/20544, currentSize=7.18 KB/7357 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 51ms, sequenceid=233, compaction requested=false 2023-08-15 16:01:04,158 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:04,556 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/4a5e796803d94960bceb543d805eaaef as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4a5e796803d94960bceb543d805eaaef 2023-08-15 16:01:04,562 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3068f0f945bfd5efbdf204bef7e4c6b0/info of 3068f0f945bfd5efbdf204bef7e4c6b0 into 4a5e796803d94960bceb543d805eaaef(size=112.3 K), total size for store is 135.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:01:04,562 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:04,562 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0., storeName=3068f0f945bfd5efbdf204bef7e4c6b0/info, priority=13, startTime=1692115264106; duration=0sec 2023-08-15 16:01:04,562 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:06,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:06,119 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-15 16:01:06,131 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=245 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/feae9f6432b543b6aebf7eef94b1f039 2023-08-15 16:01:06,137 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/feae9f6432b543b6aebf7eef94b1f039 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/feae9f6432b543b6aebf7eef94b1f039 2023-08-15 16:01:06,143 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/feae9f6432b543b6aebf7eef94b1f039, entries=8, sequenceid=245, filesize=13.0 K 2023-08-15 16:01:06,144 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=20.53 KB/21020 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 25ms, sequenceid=245, compaction requested=true 2023-08-15 16:01:06,144 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:06,144 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:06,145 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 16:01:06,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:06,145 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-08-15 16:01:06,146 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 152131 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 16:01:06,147 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 3068f0f945bfd5efbdf204bef7e4c6b0/info is initiating minor compaction (all files) 2023-08-15 16:01:06,147 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3068f0f945bfd5efbdf204bef7e4c6b0/info in TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:06,147 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4a5e796803d94960bceb543d805eaaef, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0939472c58744132818f774ffb0d0b3a, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/feae9f6432b543b6aebf7eef94b1f039] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp, totalSize=148.6 K 2023-08-15 16:01:06,147 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 4a5e796803d94960bceb543d805eaaef, keycount=104, bloomtype=ROW, size=112.3 K, encoding=NONE, compression=NONE, seqNum=212, earliestPutTs=1692115205500 2023-08-15 16:01:06,148 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 0939472c58744132818f774ffb0d0b3a, keycount=18, bloomtype=ROW, size=23.3 K, encoding=NONE, compression=NONE, seqNum=233, earliestPutTs=1692115264080 2023-08-15 16:01:06,148 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting feae9f6432b543b6aebf7eef94b1f039, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=245, earliestPutTs=1692115264107 2023-08-15 16:01:06,159 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=3068f0f945bfd5efbdf204bef7e4c6b0, server=jenkins-hbase4.apache.org,36385,1692115181760 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-08-15 16:01:06,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] ipc.CallRunner(144): callId: 243 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:49860 deadline: 1692115276159, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=3068f0f945bfd5efbdf204bef7e4c6b0, server=jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:01:06,162 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=269 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/bae1795e2c4d4c5587097704a31d10b3 2023-08-15 16:01:06,165 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3068f0f945bfd5efbdf204bef7e4c6b0#info#compaction#39 average throughput is 65.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:01:06,168 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/bae1795e2c4d4c5587097704a31d10b3 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/bae1795e2c4d4c5587097704a31d10b3 2023-08-15 16:01:06,177 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/bae1795e2c4d4c5587097704a31d10b3, entries=21, sequenceid=269, filesize=26.4 K 2023-08-15 16:01:06,178 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=8.21 KB/8408 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 33ms, sequenceid=269, compaction requested=false 2023-08-15 16:01:06,178 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:06,180 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/7707362ae2034a16b8937497d046e4b0 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7707362ae2034a16b8937497d046e4b0 2023-08-15 16:01:06,185 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3068f0f945bfd5efbdf204bef7e4c6b0/info of 3068f0f945bfd5efbdf204bef7e4c6b0 into 7707362ae2034a16b8937497d046e4b0(size=139.4 K), total size for store is 165.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:01:06,185 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:06,185 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0., storeName=3068f0f945bfd5efbdf204bef7e4c6b0/info, priority=13, startTime=1692115266144; duration=0sec 2023-08-15 16:01:06,185 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:16,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:16,253 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=9.24 KB heapSize=10.16 KB 2023-08-15 16:01:16,265 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.24 KB at sequenceid=282 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/2656d45ab92040b2abc6c7adf34b3491 2023-08-15 16:01:16,271 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/2656d45ab92040b2abc6c7adf34b3491 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/2656d45ab92040b2abc6c7adf34b3491 2023-08-15 16:01:16,276 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/2656d45ab92040b2abc6c7adf34b3491, entries=9, sequenceid=282, filesize=14.0 K 2023-08-15 16:01:16,277 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~9.24 KB/9459, heapSize ~10.15 KB/10392, currentSize=1.03 KB/1051 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 24ms, sequenceid=282, compaction requested=true 2023-08-15 16:01:16,277 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:16,277 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:16,277 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 16:01:16,278 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 184048 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 16:01:16,279 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 3068f0f945bfd5efbdf204bef7e4c6b0/info is initiating minor compaction (all files) 2023-08-15 16:01:16,279 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3068f0f945bfd5efbdf204bef7e4c6b0/info in TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:16,279 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7707362ae2034a16b8937497d046e4b0, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/bae1795e2c4d4c5587097704a31d10b3, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/2656d45ab92040b2abc6c7adf34b3491] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp, totalSize=179.7 K 2023-08-15 16:01:16,279 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 7707362ae2034a16b8937497d046e4b0, keycount=130, bloomtype=ROW, size=139.4 K, encoding=NONE, compression=NONE, seqNum=245, earliestPutTs=1692115205500 2023-08-15 16:01:16,280 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting bae1795e2c4d4c5587097704a31d10b3, keycount=21, bloomtype=ROW, size=26.4 K, encoding=NONE, compression=NONE, seqNum=269, earliestPutTs=1692115266120 2023-08-15 16:01:16,280 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting 2656d45ab92040b2abc6c7adf34b3491, keycount=9, bloomtype=ROW, size=14.0 K, encoding=NONE, compression=NONE, seqNum=282, earliestPutTs=1692115266146 2023-08-15 16:01:16,291 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3068f0f945bfd5efbdf204bef7e4c6b0#info#compaction#41 average throughput is 53.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:01:16,304 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/ac04b9e09eb8446ba61e6c474a692692 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/ac04b9e09eb8446ba61e6c474a692692 2023-08-15 16:01:16,310 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3068f0f945bfd5efbdf204bef7e4c6b0/info of 3068f0f945bfd5efbdf204bef7e4c6b0 into ac04b9e09eb8446ba61e6c474a692692(size=170.3 K), total size for store is 170.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:01:16,310 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:16,310 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0., storeName=3068f0f945bfd5efbdf204bef7e4c6b0/info, priority=13, startTime=1692115276277; duration=0sec 2023-08-15 16:01:16,310 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:18,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:18,263 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-08-15 16:01:18,275 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=294 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/eedb453f0beb4d9dbf4ad5ca14c60bb4 2023-08-15 16:01:18,285 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/eedb453f0beb4d9dbf4ad5ca14c60bb4 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/eedb453f0beb4d9dbf4ad5ca14c60bb4 2023-08-15 16:01:18,291 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/eedb453f0beb4d9dbf4ad5ca14c60bb4, entries=8, sequenceid=294, filesize=13.0 K 2023-08-15 16:01:18,292 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 29ms, sequenceid=294, compaction requested=false 2023-08-15 16:01:18,292 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:18,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36385] regionserver.HRegion(9158): Flush requested on 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:18,293 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-08-15 16:01:18,316 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=319 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/a182d04b41e84a4c84ba66c3d85007d4 2023-08-15 16:01:18,322 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/a182d04b41e84a4c84ba66c3d85007d4 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/a182d04b41e84a4c84ba66c3d85007d4 2023-08-15 16:01:18,328 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/a182d04b41e84a4c84ba66c3d85007d4, entries=22, sequenceid=319, filesize=27.4 K 2023-08-15 16:01:18,329 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=3.08 KB/3153 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 36ms, sequenceid=319, compaction requested=true 2023-08-15 16:01:18,329 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:18,329 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:18,329 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-08-15 16:01:18,330 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 215778 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-08-15 16:01:18,330 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1912): 3068f0f945bfd5efbdf204bef7e4c6b0/info is initiating minor compaction (all files) 2023-08-15 16:01:18,331 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3068f0f945bfd5efbdf204bef7e4c6b0/info in TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:18,331 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/ac04b9e09eb8446ba61e6c474a692692, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/eedb453f0beb4d9dbf4ad5ca14c60bb4, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/a182d04b41e84a4c84ba66c3d85007d4] into tmpdir=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp, totalSize=210.7 K 2023-08-15 16:01:18,331 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting ac04b9e09eb8446ba61e6c474a692692, keycount=160, bloomtype=ROW, size=170.3 K, encoding=NONE, compression=NONE, seqNum=282, earliestPutTs=1692115205500 2023-08-15 16:01:18,332 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting eedb453f0beb4d9dbf4ad5ca14c60bb4, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=294, earliestPutTs=1692115276254 2023-08-15 16:01:18,332 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] compactions.Compactor(207): Compacting a182d04b41e84a4c84ba66c3d85007d4, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=319, earliestPutTs=1692115278264 2023-08-15 16:01:18,352 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3068f0f945bfd5efbdf204bef7e4c6b0#info#compaction#44 average throughput is 63.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-15 16:01:18,368 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/5b853d84f2bd43bc84bf33b30efd206a as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/5b853d84f2bd43bc84bf33b30efd206a 2023-08-15 16:01:18,374 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3068f0f945bfd5efbdf204bef7e4c6b0/info of 3068f0f945bfd5efbdf204bef7e4c6b0 into 5b853d84f2bd43bc84bf33b30efd206a(size=201.4 K), total size for store is 201.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-15 16:01:18,374 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:18,375 INFO [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0., storeName=3068f0f945bfd5efbdf204bef7e4c6b0/info, priority=13, startTime=1692115278329; duration=0sec 2023-08-15 16:01:18,375 DEBUG [RS:0;jenkins-hbase4:36385-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-15 16:01:20,297 INFO [Listener at localhost/33883] wal.AbstractTestLogRolling(188): after writing there are 0 log files 2023-08-15 16:01:20,320 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK] 2023-08-15 16:01:20,320 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK] 2023-08-15 16:01:20,324 INFO [Listener at localhost/33883] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115182549 with entries=309, filesize=300.61 KB; new WAL /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115280298 2023-08-15 16:01:20,325 DEBUG [Listener at localhost/33883] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK], DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK]] 2023-08-15 16:01:20,325 DEBUG [Listener at localhost/33883] wal.AbstractFSWAL(716): hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115182549 is not closed yet, will try archiving it next time 2023-08-15 16:01:20,330 DEBUG [Listener at localhost/33883] regionserver.HRegion(2446): Flush status journal for 93bcc30e744bc8bbeac8634baeccb04b: 2023-08-15 16:01:20,331 INFO [Listener at localhost/33883] regionserver.HRegion(2745): Flushing 339df4b91c91c52e8aa5664386fa3b87 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-15 16:01:20,353 INFO [Listener at localhost/33883] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87/.tmp/info/523faf5952a544e19d9129c639d8cce4 2023-08-15 16:01:20,362 DEBUG [Listener at localhost/33883] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87/.tmp/info/523faf5952a544e19d9129c639d8cce4 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87/info/523faf5952a544e19d9129c639d8cce4 2023-08-15 16:01:20,369 INFO [Listener at localhost/33883] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87/info/523faf5952a544e19d9129c639d8cce4, entries=2, sequenceid=6, filesize=4.8 K 2023-08-15 16:01:20,370 INFO [Listener at localhost/33883] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 339df4b91c91c52e8aa5664386fa3b87 in 39ms, sequenceid=6, compaction requested=false 2023-08-15 16:01:20,371 DEBUG [Listener at localhost/33883] regionserver.HRegion(2446): Flush status journal for 339df4b91c91c52e8aa5664386fa3b87: 2023-08-15 16:01:20,371 INFO [Listener at localhost/33883] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.26 KB heapSize=4.19 KB 2023-08-15 16:01:20,392 INFO [Listener at localhost/33883] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.26 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/.tmp/info/36986259ec5b4258961edd58e24e0627 2023-08-15 16:01:20,400 DEBUG [Listener at localhost/33883] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/.tmp/info/36986259ec5b4258961edd58e24e0627 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/info/36986259ec5b4258961edd58e24e0627 2023-08-15 16:01:20,406 INFO [Listener at localhost/33883] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/info/36986259ec5b4258961edd58e24e0627, entries=16, sequenceid=24, filesize=7.0 K 2023-08-15 16:01:20,407 INFO [Listener at localhost/33883] regionserver.HRegion(2948): Finished flush of dataSize ~2.26 KB/2312, heapSize ~3.67 KB/3760, currentSize=0 B/0 for 1588230740 in 36ms, sequenceid=24, compaction requested=false 2023-08-15 16:01:20,408 DEBUG [Listener at localhost/33883] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-08-15 16:01:20,408 INFO [Listener at localhost/33883] regionserver.HRegion(2745): Flushing 3068f0f945bfd5efbdf204bef7e4c6b0 1/1 column families, dataSize=3.08 KB heapSize=3.55 KB 2023-08-15 16:01:20,426 INFO [Listener at localhost/33883] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.08 KB at sequenceid=326 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/72aab0c064064edfaa60786b892e56d1 2023-08-15 16:01:20,433 DEBUG [Listener at localhost/33883] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/.tmp/info/72aab0c064064edfaa60786b892e56d1 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/72aab0c064064edfaa60786b892e56d1 2023-08-15 16:01:20,439 INFO [Listener at localhost/33883] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/72aab0c064064edfaa60786b892e56d1, entries=3, sequenceid=326, filesize=7.8 K 2023-08-15 16:01:20,440 INFO [Listener at localhost/33883] regionserver.HRegion(2948): Finished flush of dataSize ~3.08 KB/3153, heapSize ~3.54 KB/3624, currentSize=0 B/0 for 3068f0f945bfd5efbdf204bef7e4c6b0 in 32ms, sequenceid=326, compaction requested=false 2023-08-15 16:01:20,440 DEBUG [Listener at localhost/33883] regionserver.HRegion(2446): Flush status journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:20,456 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK] 2023-08-15 16:01:20,456 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK] 2023-08-15 16:01:20,458 INFO [Listener at localhost/33883] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115280298 with entries=4, filesize=1.22 KB; new WAL /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115280440 2023-08-15 16:01:20,458 DEBUG [Listener at localhost/33883] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42481,DS-03d21dd4-76d0-4db8-9e77-533db29d9b76,DISK], DatanodeInfoWithStorage[127.0.0.1:36497,DS-1688ff13-43fa-4d58-8763-93e1a08ae9d0,DISK]] 2023-08-15 16:01:20,459 DEBUG [Listener at localhost/33883] wal.AbstractFSWAL(716): hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115280298 is not closed yet, will try archiving it next time 2023-08-15 16:01:20,459 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115182549 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/oldWALs/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115182549 2023-08-15 16:01:20,460 INFO [Listener at localhost/33883] hbase.Waiter(180): Waiting up to [5,000] milli-secs(wait.for.ratio=[1]) 2023-08-15 16:01:20,462 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/WALs/jenkins-hbase4.apache.org,36385,1692115181760/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115280298 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/oldWALs/jenkins-hbase4.apache.org%2C36385%2C1692115181760.1692115280298 2023-08-15 16:01:20,560 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-15 16:01:20,560 INFO [Listener at localhost/33883] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-15 16:01:20,561 DEBUG [Listener at localhost/33883] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0733519d to 127.0.0.1:50312 2023-08-15 16:01:20,561 DEBUG [Listener at localhost/33883] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:20,561 DEBUG [Listener at localhost/33883] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-15 16:01:20,561 DEBUG [Listener at localhost/33883] util.JVMClusterUtil(257): Found active master hash=612541823, stopped=false 2023-08-15 16:01:20,561 INFO [Listener at localhost/33883] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 16:01:20,562 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-15 16:01:20,562 INFO [Listener at localhost/33883] procedure2.ProcedureExecutor(629): Stopping 2023-08-15 16:01:20,562 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-15 16:01:20,563 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:20,563 DEBUG [Listener at localhost/33883] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4a14185f to 127.0.0.1:50312 2023-08-15 16:01:20,564 DEBUG [Listener at localhost/33883] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:20,564 INFO [Listener at localhost/33883] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36385,1692115181760' ***** 2023-08-15 16:01:20,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 16:01:20,564 INFO [Listener at localhost/33883] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-15 16:01:20,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 16:01:20,565 INFO [RS:0;jenkins-hbase4:36385] regionserver.HeapMemoryManager(220): Stopping 2023-08-15 16:01:20,565 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-15 16:01:20,565 INFO [RS:0;jenkins-hbase4:36385] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-15 16:01:20,565 INFO [RS:0;jenkins-hbase4:36385] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-15 16:01:20,565 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(3305): Received CLOSE for 93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:01:20,565 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(3305): Received CLOSE for 339df4b91c91c52e8aa5664386fa3b87 2023-08-15 16:01:20,565 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(3305): Received CLOSE for 3068f0f945bfd5efbdf204bef7e4c6b0 2023-08-15 16:01:20,565 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:01:20,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 93bcc30e744bc8bbeac8634baeccb04b, disabling compactions & flushes 2023-08-15 16:01:20,565 DEBUG [RS:0;jenkins-hbase4:36385] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0f7724a4 to 127.0.0.1:50312 2023-08-15 16:01:20,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:01:20,566 DEBUG [RS:0;jenkins-hbase4:36385] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:20,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:01:20,566 INFO [RS:0;jenkins-hbase4:36385] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-15 16:01:20,566 INFO [RS:0;jenkins-hbase4:36385] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-15 16:01:20,566 INFO [RS:0;jenkins-hbase4:36385] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-15 16:01:20,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. after waiting 0 ms 2023-08-15 16:01:20,566 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-15 16:01:20,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:01:20,566 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-08-15 16:01:20,566 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1478): Online Regions={93bcc30e744bc8bbeac8634baeccb04b=TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b., 339df4b91c91c52e8aa5664386fa3b87=hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87., 1588230740=hbase:meta,,1.1588230740, 3068f0f945bfd5efbdf204bef7e4c6b0=TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.} 2023-08-15 16:01:20,566 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-15 16:01:20,566 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-15 16:01:20,566 DEBUG [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1504): Waiting on 1588230740, 3068f0f945bfd5efbdf204bef7e4c6b0, 339df4b91c91c52e8aa5664386fa3b87, 93bcc30e744bc8bbeac8634baeccb04b 2023-08-15 16:01:20,566 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-15 16:01:20,566 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-15 16:01:20,566 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-15 16:01:20,568 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473->hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931-bottom] to archive 2023-08-15 16:01:20,572 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-15 16:01:20,577 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473 2023-08-15 16:01:20,578 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-08-15 16:01:20,579 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-15 16:01:20,580 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-15 16:01:20,580 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-15 16:01:20,580 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-15 16:01:20,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/93bcc30e744bc8bbeac8634baeccb04b/recovered.edits/120.seqid, newMaxSeqId=120, maxSeqId=115 2023-08-15 16:01:20,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:01:20,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 93bcc30e744bc8bbeac8634baeccb04b: 2023-08-15 16:01:20,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,,1692115207668.93bcc30e744bc8bbeac8634baeccb04b. 2023-08-15 16:01:20,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 339df4b91c91c52e8aa5664386fa3b87, disabling compactions & flushes 2023-08-15 16:01:20,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 16:01:20,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 16:01:20,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. after waiting 0 ms 2023-08-15 16:01:20,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 16:01:20,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/hbase/namespace/339df4b91c91c52e8aa5664386fa3b87/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-15 16:01:20,591 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 16:01:20,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 339df4b91c91c52e8aa5664386fa3b87: 2023-08-15 16:01:20,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692115182790.339df4b91c91c52e8aa5664386fa3b87. 2023-08-15 16:01:20,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3068f0f945bfd5efbdf204bef7e4c6b0, disabling compactions & flushes 2023-08-15 16:01:20,591 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:20,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:20,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. after waiting 0 ms 2023-08-15 16:01:20,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:20,609 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473->hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/5623bed68b8364b08ed7682e64962473/info/fa2534aec0334c239e2a126c1a995931-top, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/9681ecf3fa4347aaae55823c98318e5b, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/TestLogRolling-testLogRolling=5623bed68b8364b08ed7682e64962473-9cc92b136e2747ba824feaaa2a425b86, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/93c4ea9164d0454d82e06c62ffd2ec12, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0f97807e8ea94719b99022c70c92a921, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/dc642efc184542f39b23617899cfd419, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/1d2a51ccf3e6410591fc319b71c4bb5e, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/8dbbcd97a6ed47b2b880210901fddbf8, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/b135acdf33c6457683f140683712ed83, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7e5a3294b5a949e59c9eaf55ba821148, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4a5e796803d94960bceb543d805eaaef, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4c4c24671f0b44319904c9367262c0ac, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0939472c58744132818f774ffb0d0b3a, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7707362ae2034a16b8937497d046e4b0, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/feae9f6432b543b6aebf7eef94b1f039, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/bae1795e2c4d4c5587097704a31d10b3, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/ac04b9e09eb8446ba61e6c474a692692, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/2656d45ab92040b2abc6c7adf34b3491, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/eedb453f0beb4d9dbf4ad5ca14c60bb4, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/a182d04b41e84a4c84ba66c3d85007d4] to archive 2023-08-15 16:01:20,611 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-15 16:01:20,613 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/fa2534aec0334c239e2a126c1a995931.5623bed68b8364b08ed7682e64962473 2023-08-15 16:01:20,615 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/9681ecf3fa4347aaae55823c98318e5b to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/9681ecf3fa4347aaae55823c98318e5b 2023-08-15 16:01:20,616 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/TestLogRolling-testLogRolling=5623bed68b8364b08ed7682e64962473-9cc92b136e2747ba824feaaa2a425b86 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/TestLogRolling-testLogRolling=5623bed68b8364b08ed7682e64962473-9cc92b136e2747ba824feaaa2a425b86 2023-08-15 16:01:20,618 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/93c4ea9164d0454d82e06c62ffd2ec12 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/93c4ea9164d0454d82e06c62ffd2ec12 2023-08-15 16:01:20,620 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0f97807e8ea94719b99022c70c92a921 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0f97807e8ea94719b99022c70c92a921 2023-08-15 16:01:20,621 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/dc642efc184542f39b23617899cfd419 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/dc642efc184542f39b23617899cfd419 2023-08-15 16:01:20,623 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/1d2a51ccf3e6410591fc319b71c4bb5e to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/1d2a51ccf3e6410591fc319b71c4bb5e 2023-08-15 16:01:20,624 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/8dbbcd97a6ed47b2b880210901fddbf8 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/8dbbcd97a6ed47b2b880210901fddbf8 2023-08-15 16:01:20,626 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/b135acdf33c6457683f140683712ed83 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/b135acdf33c6457683f140683712ed83 2023-08-15 16:01:20,627 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7e5a3294b5a949e59c9eaf55ba821148 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7e5a3294b5a949e59c9eaf55ba821148 2023-08-15 16:01:20,629 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4a5e796803d94960bceb543d805eaaef to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4a5e796803d94960bceb543d805eaaef 2023-08-15 16:01:20,630 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4c4c24671f0b44319904c9367262c0ac to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/4c4c24671f0b44319904c9367262c0ac 2023-08-15 16:01:20,632 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0939472c58744132818f774ffb0d0b3a to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/0939472c58744132818f774ffb0d0b3a 2023-08-15 16:01:20,633 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7707362ae2034a16b8937497d046e4b0 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/7707362ae2034a16b8937497d046e4b0 2023-08-15 16:01:20,634 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/feae9f6432b543b6aebf7eef94b1f039 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/feae9f6432b543b6aebf7eef94b1f039 2023-08-15 16:01:20,636 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/bae1795e2c4d4c5587097704a31d10b3 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/bae1795e2c4d4c5587097704a31d10b3 2023-08-15 16:01:20,637 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/ac04b9e09eb8446ba61e6c474a692692 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/ac04b9e09eb8446ba61e6c474a692692 2023-08-15 16:01:20,638 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/2656d45ab92040b2abc6c7adf34b3491 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/2656d45ab92040b2abc6c7adf34b3491 2023-08-15 16:01:20,640 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/eedb453f0beb4d9dbf4ad5ca14c60bb4 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/eedb453f0beb4d9dbf4ad5ca14c60bb4 2023-08-15 16:01:20,641 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/a182d04b41e84a4c84ba66c3d85007d4 to hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/archive/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/info/a182d04b41e84a4c84ba66c3d85007d4 2023-08-15 16:01:20,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/data/default/TestLogRolling-testLogRolling/3068f0f945bfd5efbdf204bef7e4c6b0/recovered.edits/329.seqid, newMaxSeqId=329, maxSeqId=115 2023-08-15 16:01:20,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:20,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3068f0f945bfd5efbdf204bef7e4c6b0: 2023-08-15 16:01:20,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,row0064,1692115207668.3068f0f945bfd5efbdf204bef7e4c6b0. 2023-08-15 16:01:20,766 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36385,1692115181760; all regions closed. 2023-08-15 16:01:20,773 DEBUG [RS:0;jenkins-hbase4:36385] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/oldWALs 2023-08-15 16:01:20,774 INFO [RS:0;jenkins-hbase4:36385] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36385%2C1692115181760.meta:.meta(num 1692115182734) 2023-08-15 16:01:20,779 DEBUG [RS:0;jenkins-hbase4:36385] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/oldWALs 2023-08-15 16:01:20,779 INFO [RS:0;jenkins-hbase4:36385] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36385%2C1692115181760:(num 1692115280440) 2023-08-15 16:01:20,779 DEBUG [RS:0;jenkins-hbase4:36385] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:20,779 INFO [RS:0;jenkins-hbase4:36385] regionserver.LeaseManager(133): Closed leases 2023-08-15 16:01:20,779 INFO [RS:0;jenkins-hbase4:36385] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-15 16:01:20,780 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-15 16:01:20,780 INFO [RS:0;jenkins-hbase4:36385] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36385 2023-08-15 16:01:20,783 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36385,1692115181760 2023-08-15 16:01:20,783 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 16:01:20,783 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 16:01:20,785 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36385,1692115181760] 2023-08-15 16:01:20,785 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36385,1692115181760; numProcessing=1 2023-08-15 16:01:20,787 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36385,1692115181760 already deleted, retry=false 2023-08-15 16:01:20,787 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36385,1692115181760 expired; onlineServers=0 2023-08-15 16:01:20,787 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34795,1692115181709' ***** 2023-08-15 16:01:20,787 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-15 16:01:20,787 DEBUG [M:0;jenkins-hbase4:34795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@20bd2537, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-15 16:01:20,787 INFO [M:0;jenkins-hbase4:34795] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34795,1692115181709 2023-08-15 16:01:20,787 INFO [M:0;jenkins-hbase4:34795] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34795,1692115181709; all regions closed. 2023-08-15 16:01:20,787 DEBUG [M:0;jenkins-hbase4:34795] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:20,788 DEBUG [M:0;jenkins-hbase4:34795] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-15 16:01:20,788 DEBUG [M:0;jenkins-hbase4:34795] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-15 16:01:20,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115182357] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115182357,5,FailOnTimeoutGroup] 2023-08-15 16:01:20,788 INFO [M:0;jenkins-hbase4:34795] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-15 16:01:20,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115182356] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115182356,5,FailOnTimeoutGroup] 2023-08-15 16:01:20,788 INFO [M:0;jenkins-hbase4:34795] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-15 16:01:20,788 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-15 16:01:20,789 INFO [M:0;jenkins-hbase4:34795] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-15 16:01:20,789 DEBUG [M:0;jenkins-hbase4:34795] master.HMaster(1512): Stopping service threads 2023-08-15 16:01:20,789 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-15 16:01:20,789 INFO [M:0;jenkins-hbase4:34795] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-15 16:01:20,789 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:20,789 ERROR [M:0;jenkins-hbase4:34795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-15 16:01:20,790 INFO [M:0;jenkins-hbase4:34795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-15 16:01:20,790 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-15 16:01:20,790 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 16:01:20,790 DEBUG [M:0;jenkins-hbase4:34795] zookeeper.ZKUtil(398): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-15 16:01:20,790 WARN [M:0;jenkins-hbase4:34795] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-15 16:01:20,790 INFO [M:0;jenkins-hbase4:34795] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-15 16:01:20,791 INFO [M:0;jenkins-hbase4:34795] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-15 16:01:20,791 DEBUG [M:0;jenkins-hbase4:34795] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-15 16:01:20,791 INFO [M:0;jenkins-hbase4:34795] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:20,791 DEBUG [M:0;jenkins-hbase4:34795] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:20,791 DEBUG [M:0;jenkins-hbase4:34795] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-15 16:01:20,791 DEBUG [M:0;jenkins-hbase4:34795] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:20,792 INFO [M:0;jenkins-hbase4:34795] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.70 KB heapSize=78.42 KB 2023-08-15 16:01:20,810 INFO [M:0;jenkins-hbase4:34795] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.70 KB at sequenceid=160 (bloomFilter=true), to=hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a2da17a0de964f228d690b3fa6a1baf6 2023-08-15 16:01:20,819 INFO [M:0;jenkins-hbase4:34795] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2da17a0de964f228d690b3fa6a1baf6 2023-08-15 16:01:20,820 DEBUG [M:0;jenkins-hbase4:34795] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a2da17a0de964f228d690b3fa6a1baf6 as hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a2da17a0de964f228d690b3fa6a1baf6 2023-08-15 16:01:20,826 INFO [M:0;jenkins-hbase4:34795] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2da17a0de964f228d690b3fa6a1baf6 2023-08-15 16:01:20,826 INFO [M:0;jenkins-hbase4:34795] regionserver.HStore(1080): Added hdfs://localhost:45523/user/jenkins/test-data/dfe2d54f-60d4-1b5c-db9e-51fad8c4079e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a2da17a0de964f228d690b3fa6a1baf6, entries=18, sequenceid=160, filesize=6.9 K 2023-08-15 16:01:20,827 INFO [M:0;jenkins-hbase4:34795] regionserver.HRegion(2948): Finished flush of dataSize ~64.70 KB/66256, heapSize ~78.41 KB/80288, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 36ms, sequenceid=160, compaction requested=false 2023-08-15 16:01:20,828 INFO [M:0;jenkins-hbase4:34795] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:20,828 DEBUG [M:0;jenkins-hbase4:34795] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 16:01:20,836 INFO [M:0;jenkins-hbase4:34795] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-15 16:01:20,836 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-15 16:01:20,837 INFO [M:0;jenkins-hbase4:34795] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34795 2023-08-15 16:01:20,839 DEBUG [M:0;jenkins-hbase4:34795] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,34795,1692115181709 already deleted, retry=false 2023-08-15 16:01:20,886 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 16:01:20,886 INFO [RS:0;jenkins-hbase4:36385] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36385,1692115181760; zookeeper connection closed. 2023-08-15 16:01:20,886 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): regionserver:36385-0x10010abd81a0001, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 16:01:20,887 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2bb601d4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2bb601d4 2023-08-15 16:01:20,887 INFO [Listener at localhost/33883] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-15 16:01:20,986 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 16:01:20,986 INFO [M:0;jenkins-hbase4:34795] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34795,1692115181709; zookeeper connection closed. 2023-08-15 16:01:20,986 DEBUG [Listener at localhost/33883-EventThread] zookeeper.ZKWatcher(600): master:34795-0x10010abd81a0000, quorum=127.0.0.1:50312, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 16:01:20,987 WARN [Listener at localhost/33883] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 16:01:20,992 INFO [Listener at localhost/33883] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 16:01:21,000 WARN [BP-1561502560-172.31.14.131-1692115180976 heartbeating to localhost/127.0.0.1:45523] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 16:01:21,000 WARN [BP-1561502560-172.31.14.131-1692115180976 heartbeating to localhost/127.0.0.1:45523] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1561502560-172.31.14.131-1692115180976 (Datanode Uuid fabdd565-92ff-4093-bd2c-eaa59afd4719) service to localhost/127.0.0.1:45523 2023-08-15 16:01:21,001 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/cluster_505db672-224e-5593-010e-aed0c4218503/dfs/data/data3/current/BP-1561502560-172.31.14.131-1692115180976] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 16:01:21,002 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/cluster_505db672-224e-5593-010e-aed0c4218503/dfs/data/data4/current/BP-1561502560-172.31.14.131-1692115180976] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 16:01:21,003 WARN [Listener at localhost/33883] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 16:01:21,011 INFO [Listener at localhost/33883] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 16:01:21,118 WARN [BP-1561502560-172.31.14.131-1692115180976 heartbeating to localhost/127.0.0.1:45523] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 16:01:21,118 WARN [BP-1561502560-172.31.14.131-1692115180976 heartbeating to localhost/127.0.0.1:45523] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1561502560-172.31.14.131-1692115180976 (Datanode Uuid addd2e78-719f-4683-ba13-a7a9b0d9e679) service to localhost/127.0.0.1:45523 2023-08-15 16:01:21,119 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/cluster_505db672-224e-5593-010e-aed0c4218503/dfs/data/data1/current/BP-1561502560-172.31.14.131-1692115180976] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 16:01:21,119 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/cluster_505db672-224e-5593-010e-aed0c4218503/dfs/data/data2/current/BP-1561502560-172.31.14.131-1692115180976] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 16:01:21,133 INFO [Listener at localhost/33883] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 16:01:21,250 INFO [Listener at localhost/33883] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-15 16:01:21,309 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-15 16:01:21,325 INFO [Listener at localhost/33883] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=76 (was 65) - Thread LEAK? -, OpenFileDescriptor=471 (was 430) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=147 (was 91) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=771 (was 685) - AvailableMemoryMB LEAK? - 2023-08-15 16:01:21,334 INFO [Listener at localhost/33883] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=77, OpenFileDescriptor=471, MaxFileDescriptor=60000, SystemLoadAverage=147, ProcessCount=169, AvailableMemoryMB=771 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/hadoop.log.dir so I do NOT create it in target/test-data/3add03e2-1a34-241c-d053-a37975dec82d 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ce13915c-c42d-a01c-aa09-2e7808a2ee4a/hadoop.tmp.dir so I do NOT create it in target/test-data/3add03e2-1a34-241c-d053-a37975dec82d 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/cluster_eb197253-fe0d-4177-c180-83b80fdd7673, deleteOnExit=true 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/test.cache.data in system properties and HBase conf 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/hadoop.tmp.dir in system properties and HBase conf 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/hadoop.log.dir in system properties and HBase conf 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-15 16:01:21,335 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-15 16:01:21,336 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-15 16:01:21,336 DEBUG [Listener at localhost/33883] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-15 16:01:21,336 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-15 16:01:21,336 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-15 16:01:21,336 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-15 16:01:21,336 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-15 16:01:21,336 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-15 16:01:21,336 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-15 16:01:21,336 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-15 16:01:21,337 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-15 16:01:21,337 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-15 16:01:21,337 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/nfs.dump.dir in system properties and HBase conf 2023-08-15 16:01:21,337 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/java.io.tmpdir in system properties and HBase conf 2023-08-15 16:01:21,337 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-15 16:01:21,338 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-15 16:01:21,338 INFO [Listener at localhost/33883] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-15 16:01:21,344 WARN [Listener at localhost/33883] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-15 16:01:21,345 WARN [Listener at localhost/33883] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-15 16:01:21,394 WARN [Listener at localhost/33883] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 16:01:21,396 INFO [Listener at localhost/33883] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 16:01:21,401 INFO [Listener at localhost/33883] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/java.io.tmpdir/Jetty_localhost_43477_hdfs____.fs8sv5/webapp 2023-08-15 16:01:21,506 INFO [Listener at localhost/33883] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43477 2023-08-15 16:01:21,586 WARN [Listener at localhost/33883] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-15 16:01:21,588 WARN [Listener at localhost/33883] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-15 16:01:21,635 WARN [Listener at localhost/43307] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 16:01:21,662 WARN [Listener at localhost/43307] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 16:01:21,664 WARN [Listener at localhost/43307] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 16:01:21,665 INFO [Listener at localhost/43307] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 16:01:21,670 INFO [Listener at localhost/43307] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/java.io.tmpdir/Jetty_localhost_39201_datanode____ut7pbo/webapp 2023-08-15 16:01:21,764 INFO [Listener at localhost/43307] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39201 2023-08-15 16:01:21,773 WARN [Listener at localhost/37731] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 16:01:21,789 WARN [Listener at localhost/37731] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-15 16:01:21,791 WARN [Listener at localhost/37731] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-15 16:01:21,792 INFO [Listener at localhost/37731] log.Slf4jLog(67): jetty-6.1.26 2023-08-15 16:01:21,799 INFO [Listener at localhost/37731] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/java.io.tmpdir/Jetty_localhost_41385_datanode____.bnnlc6/webapp 2023-08-15 16:01:21,872 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb173bb24ed93dc80: Processing first storage report for DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f from datanode 746804e0-8861-4494-a303-587f29c10b9d 2023-08-15 16:01:21,872 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb173bb24ed93dc80: from storage DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f node DatanodeRegistration(127.0.0.1:39463, datanodeUuid=746804e0-8861-4494-a303-587f29c10b9d, infoPort=37397, infoSecurePort=0, ipcPort=37731, storageInfo=lv=-57;cid=testClusterID;nsid=816106834;c=1692115281352), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 16:01:21,872 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb173bb24ed93dc80: Processing first storage report for DS-93ed5bee-e245-4c72-b85b-c77cf589d9f1 from datanode 746804e0-8861-4494-a303-587f29c10b9d 2023-08-15 16:01:21,872 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb173bb24ed93dc80: from storage DS-93ed5bee-e245-4c72-b85b-c77cf589d9f1 node DatanodeRegistration(127.0.0.1:39463, datanodeUuid=746804e0-8861-4494-a303-587f29c10b9d, infoPort=37397, infoSecurePort=0, ipcPort=37731, storageInfo=lv=-57;cid=testClusterID;nsid=816106834;c=1692115281352), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 16:01:21,904 INFO [Listener at localhost/37731] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41385 2023-08-15 16:01:21,912 WARN [Listener at localhost/41601] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-15 16:01:22,009 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa8b004d11cfb431a: Processing first storage report for DS-61e70227-eee4-4aba-81c2-455547626ad6 from datanode 5bf794f4-e12b-4608-966f-fba7b4b7519a 2023-08-15 16:01:22,009 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa8b004d11cfb431a: from storage DS-61e70227-eee4-4aba-81c2-455547626ad6 node DatanodeRegistration(127.0.0.1:32867, datanodeUuid=5bf794f4-e12b-4608-966f-fba7b4b7519a, infoPort=35135, infoSecurePort=0, ipcPort=41601, storageInfo=lv=-57;cid=testClusterID;nsid=816106834;c=1692115281352), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 16:01:22,010 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa8b004d11cfb431a: Processing first storage report for DS-d56d76d3-cbe5-43ae-b074-53cac76bcde9 from datanode 5bf794f4-e12b-4608-966f-fba7b4b7519a 2023-08-15 16:01:22,010 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa8b004d11cfb431a: from storage DS-d56d76d3-cbe5-43ae-b074-53cac76bcde9 node DatanodeRegistration(127.0.0.1:32867, datanodeUuid=5bf794f4-e12b-4608-966f-fba7b4b7519a, infoPort=35135, infoSecurePort=0, ipcPort=41601, storageInfo=lv=-57;cid=testClusterID;nsid=816106834;c=1692115281352), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-15 16:01:22,020 DEBUG [Listener at localhost/41601] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d 2023-08-15 16:01:22,023 INFO [Listener at localhost/41601] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/cluster_eb197253-fe0d-4177-c180-83b80fdd7673/zookeeper_0, clientPort=51285, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/cluster_eb197253-fe0d-4177-c180-83b80fdd7673/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/cluster_eb197253-fe0d-4177-c180-83b80fdd7673/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-15 16:01:22,024 INFO [Listener at localhost/41601] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51285 2023-08-15 16:01:22,025 INFO [Listener at localhost/41601] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 16:01:22,026 INFO [Listener at localhost/41601] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 16:01:22,043 INFO [Listener at localhost/41601] util.FSUtils(471): Created version file at hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba with version=8 2023-08-15 16:01:22,043 INFO [Listener at localhost/41601] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39847/user/jenkins/test-data/828c38c4-c6dc-8820-0179-7309786db528/hbase-staging 2023-08-15 16:01:22,046 INFO [Listener at localhost/41601] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-15 16:01:22,046 INFO [Listener at localhost/41601] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 16:01:22,046 INFO [Listener at localhost/41601] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-15 16:01:22,046 INFO [Listener at localhost/41601] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-15 16:01:22,046 INFO [Listener at localhost/41601] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 16:01:22,046 INFO [Listener at localhost/41601] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-15 16:01:22,046 INFO [Listener at localhost/41601] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-15 16:01:22,048 INFO [Listener at localhost/41601] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39597 2023-08-15 16:01:22,049 INFO [Listener at localhost/41601] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 16:01:22,050 INFO [Listener at localhost/41601] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 16:01:22,051 INFO [Listener at localhost/41601] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39597 connecting to ZooKeeper ensemble=127.0.0.1:51285 2023-08-15 16:01:22,060 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:395970x0, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-15 16:01:22,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39597-0x10010ad600d0000 connected 2023-08-15 16:01:22,096 DEBUG [Listener at localhost/41601] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 16:01:22,097 DEBUG [Listener at localhost/41601] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 16:01:22,097 DEBUG [Listener at localhost/41601] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-15 16:01:22,101 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39597 2023-08-15 16:01:22,103 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39597 2023-08-15 16:01:22,103 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39597 2023-08-15 16:01:22,106 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39597 2023-08-15 16:01:22,107 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39597 2023-08-15 16:01:22,107 INFO [Listener at localhost/41601] master.HMaster(444): hbase.rootdir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba, hbase.cluster.distributed=false 2023-08-15 16:01:22,123 INFO [Listener at localhost/41601] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-15 16:01:22,124 INFO [Listener at localhost/41601] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 16:01:22,124 INFO [Listener at localhost/41601] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-15 16:01:22,124 INFO [Listener at localhost/41601] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-15 16:01:22,124 INFO [Listener at localhost/41601] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-15 16:01:22,124 INFO [Listener at localhost/41601] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-15 16:01:22,124 INFO [Listener at localhost/41601] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-15 16:01:22,128 INFO [Listener at localhost/41601] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40443 2023-08-15 16:01:22,129 INFO [Listener at localhost/41601] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-15 16:01:22,132 DEBUG [Listener at localhost/41601] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-15 16:01:22,133 INFO [Listener at localhost/41601] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 16:01:22,134 INFO [Listener at localhost/41601] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 16:01:22,136 INFO [Listener at localhost/41601] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40443 connecting to ZooKeeper ensemble=127.0.0.1:51285 2023-08-15 16:01:22,142 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): regionserver:404430x0, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-15 16:01:22,143 DEBUG [Listener at localhost/41601] zookeeper.ZKUtil(164): regionserver:404430x0, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 16:01:22,144 DEBUG [Listener at localhost/41601] zookeeper.ZKUtil(164): regionserver:404430x0, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 16:01:22,152 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40443-0x10010ad600d0001 connected 2023-08-15 16:01:22,152 DEBUG [Listener at localhost/41601] zookeeper.ZKUtil(164): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-15 16:01:22,152 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40443 2023-08-15 16:01:22,156 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40443 2023-08-15 16:01:22,157 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40443 2023-08-15 16:01:22,158 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40443 2023-08-15 16:01:22,158 DEBUG [Listener at localhost/41601] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40443 2023-08-15 16:01:22,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:22,163 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-15 16:01:22,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:22,165 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-15 16:01:22,165 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-15 16:01:22,165 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:22,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-15 16:01:22,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39597,1692115282045 from backup master directory 2023-08-15 16:01:22,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-15 16:01:22,174 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:22,174 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-15 16:01:22,174 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-15 16:01:22,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:22,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/hbase.id with ID: ec062cba-1541-4262-9f51-be5b0119c539 2023-08-15 16:01:22,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 16:01:22,203 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:22,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4e49cb05 to 127.0.0.1:51285 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 16:01:22,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17da3e7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 16:01:22,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-15 16:01:22,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-15 16:01:22,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 16:01:22,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store-tmp 2023-08-15 16:01:22,231 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 16:01:22,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-15 16:01:22,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:22,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:22,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-15 16:01:22,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:22,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:22,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 16:01:22,232 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/WALs/jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:22,235 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39597%2C1692115282045, suffix=, logDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/WALs/jenkins-hbase4.apache.org,39597,1692115282045, archiveDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/oldWALs, maxLogs=10 2023-08-15 16:01:22,250 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK] 2023-08-15 16:01:22,250 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK] 2023-08-15 16:01:22,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/WALs/jenkins-hbase4.apache.org,39597,1692115282045/jenkins-hbase4.apache.org%2C39597%2C1692115282045.1692115282235 2023-08-15 16:01:22,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK], DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK]] 2023-08-15 16:01:22,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-15 16:01:22,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 16:01:22,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 16:01:22,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 16:01:22,255 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-15 16:01:22,257 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-15 16:01:22,257 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-15 16:01:22,258 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:01:22,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-15 16:01:22,259 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-15 16:01:22,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-15 16:01:22,263 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 16:01:22,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=820563, jitterRate=0.043399810791015625}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 16:01:22,264 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 16:01:22,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-15 16:01:22,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-08-15 16:01:22,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-08-15 16:01:22,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-15 16:01:22,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-15 16:01:22,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 0 msec 2023-08-15 16:01:22,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-15 16:01:22,267 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-15 16:01:22,268 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-15 16:01:22,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-15 16:01:22,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-15 16:01:22,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-15 16:01:22,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-15 16:01:22,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-15 16:01:22,282 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:22,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-15 16:01:22,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-15 16:01:22,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-15 16:01:22,286 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-15 16:01:22,286 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-15 16:01:22,286 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:22,287 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39597,1692115282045, sessionid=0x10010ad600d0000, setting cluster-up flag (Was=false) 2023-08-15 16:01:22,290 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:22,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-15 16:01:22,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:22,300 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:22,304 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-15 16:01:22,305 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:22,306 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/.hbase-snapshot/.tmp 2023-08-15 16:01:22,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-15 16:01:22,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 16:01:22,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 16:01:22,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 16:01:22,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-15 16:01:22,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-15 16:01:22,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-15 16:01:22,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,314 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692115312314 2023-08-15 16:01:22,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-15 16:01:22,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-15 16:01:22,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-15 16:01:22,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-15 16:01:22,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-15 16:01:22,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-15 16:01:22,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,316 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-15 16:01:22,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-15 16:01:22,316 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-15 16:01:22,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-15 16:01:22,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-15 16:01:22,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-15 16:01:22,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-15 16:01:22,318 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-15 16:01:22,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115282317,5,FailOnTimeoutGroup] 2023-08-15 16:01:22,324 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115282318,5,FailOnTimeoutGroup] 2023-08-15 16:01:22,324 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-15 16:01:22,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,340 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-15 16:01:22,341 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-15 16:01:22,341 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba 2023-08-15 16:01:22,352 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 16:01:22,353 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-15 16:01:22,355 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/info 2023-08-15 16:01:22,356 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-15 16:01:22,357 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:01:22,357 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-15 16:01:22,358 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/rep_barrier 2023-08-15 16:01:22,359 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-15 16:01:22,360 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:01:22,360 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(951): ClusterId : ec062cba-1541-4262-9f51-be5b0119c539 2023-08-15 16:01:22,361 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-15 16:01:22,361 DEBUG [RS:0;jenkins-hbase4:40443] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-15 16:01:22,363 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/table 2023-08-15 16:01:22,363 DEBUG [RS:0;jenkins-hbase4:40443] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-15 16:01:22,363 DEBUG [RS:0;jenkins-hbase4:40443] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-15 16:01:22,363 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-15 16:01:22,364 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:01:22,365 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740 2023-08-15 16:01:22,365 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740 2023-08-15 16:01:22,366 DEBUG [RS:0;jenkins-hbase4:40443] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-15 16:01:22,367 DEBUG [RS:0;jenkins-hbase4:40443] zookeeper.ReadOnlyZKClient(139): Connect 0x04ac5d37 to 127.0.0.1:51285 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 16:01:22,370 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-15 16:01:22,372 DEBUG [RS:0;jenkins-hbase4:40443] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a6bb3d2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 16:01:22,372 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-15 16:01:22,372 DEBUG [RS:0;jenkins-hbase4:40443] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e543647, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-15 16:01:22,377 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 16:01:22,378 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=716009, jitterRate=-0.08954840898513794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-15 16:01:22,378 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-15 16:01:22,378 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-15 16:01:22,378 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-15 16:01:22,378 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-15 16:01:22,379 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 1 ms 2023-08-15 16:01:22,379 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-15 16:01:22,381 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-15 16:01:22,381 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-15 16:01:22,382 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40443 2023-08-15 16:01:22,382 INFO [RS:0;jenkins-hbase4:40443] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-15 16:01:22,382 INFO [RS:0;jenkins-hbase4:40443] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-15 16:01:22,382 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1022): About to register with Master. 2023-08-15 16:01:22,382 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-15 16:01:22,382 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-15 16:01:22,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-15 16:01:22,382 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39597,1692115282045 with isa=jenkins-hbase4.apache.org/172.31.14.131:40443, startcode=1692115282123 2023-08-15 16:01:22,383 DEBUG [RS:0;jenkins-hbase4:40443] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-15 16:01:22,384 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-15 16:01:22,386 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38135, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-08-15 16:01:22,387 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-15 16:01:22,388 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39597] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:22,388 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba 2023-08-15 16:01:22,388 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:43307 2023-08-15 16:01:22,388 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-15 16:01:22,390 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 16:01:22,390 DEBUG [RS:0;jenkins-hbase4:40443] zookeeper.ZKUtil(162): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:22,390 WARN [RS:0;jenkins-hbase4:40443] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-15 16:01:22,390 INFO [RS:0;jenkins-hbase4:40443] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 16:01:22,390 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1948): logDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:22,395 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40443,1692115282123] 2023-08-15 16:01:22,401 DEBUG [RS:0;jenkins-hbase4:40443] zookeeper.ZKUtil(162): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:22,403 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-15 16:01:22,403 INFO [RS:0;jenkins-hbase4:40443] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-15 16:01:22,409 INFO [RS:0;jenkins-hbase4:40443] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-15 16:01:22,409 INFO [RS:0;jenkins-hbase4:40443] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-15 16:01:22,409 INFO [RS:0;jenkins-hbase4:40443] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,410 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-15 16:01:22,411 INFO [RS:0;jenkins-hbase4:40443] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,412 DEBUG [RS:0;jenkins-hbase4:40443] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-15 16:01:22,416 INFO [RS:0;jenkins-hbase4:40443] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,416 INFO [RS:0;jenkins-hbase4:40443] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,416 INFO [RS:0;jenkins-hbase4:40443] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,430 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-15 16:01:22,431 INFO [RS:0;jenkins-hbase4:40443] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-15 16:01:22,431 INFO [RS:0;jenkins-hbase4:40443] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40443,1692115282123-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,443 INFO [RS:0;jenkins-hbase4:40443] regionserver.Replication(203): jenkins-hbase4.apache.org,40443,1692115282123 started 2023-08-15 16:01:22,443 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40443,1692115282123, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40443, sessionid=0x10010ad600d0001 2023-08-15 16:01:22,443 DEBUG [RS:0;jenkins-hbase4:40443] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-15 16:01:22,443 DEBUG [RS:0;jenkins-hbase4:40443] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:22,443 DEBUG [RS:0;jenkins-hbase4:40443] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40443,1692115282123' 2023-08-15 16:01:22,443 DEBUG [RS:0;jenkins-hbase4:40443] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-15 16:01:22,444 DEBUG [RS:0;jenkins-hbase4:40443] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-15 16:01:22,444 DEBUG [RS:0;jenkins-hbase4:40443] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-15 16:01:22,444 DEBUG [RS:0;jenkins-hbase4:40443] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-15 16:01:22,444 DEBUG [RS:0;jenkins-hbase4:40443] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:22,444 DEBUG [RS:0;jenkins-hbase4:40443] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40443,1692115282123' 2023-08-15 16:01:22,444 DEBUG [RS:0;jenkins-hbase4:40443] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-15 16:01:22,445 DEBUG [RS:0;jenkins-hbase4:40443] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-15 16:01:22,445 DEBUG [RS:0;jenkins-hbase4:40443] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-15 16:01:22,445 INFO [RS:0;jenkins-hbase4:40443] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-15 16:01:22,445 INFO [RS:0;jenkins-hbase4:40443] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-15 16:01:22,537 DEBUG [jenkins-hbase4:39597] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-08-15 16:01:22,538 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40443,1692115282123, state=OPENING 2023-08-15 16:01:22,539 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-15 16:01:22,542 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:22,542 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-15 16:01:22,542 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40443,1692115282123}] 2023-08-15 16:01:22,547 INFO [RS:0;jenkins-hbase4:40443] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40443%2C1692115282123, suffix=, logDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/jenkins-hbase4.apache.org,40443,1692115282123, archiveDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/oldWALs, maxLogs=32 2023-08-15 16:01:22,567 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK] 2023-08-15 16:01:22,567 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK] 2023-08-15 16:01:22,569 INFO [RS:0;jenkins-hbase4:40443] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/jenkins-hbase4.apache.org,40443,1692115282123/jenkins-hbase4.apache.org%2C40443%2C1692115282123.1692115282548 2023-08-15 16:01:22,569 DEBUG [RS:0;jenkins-hbase4:40443] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK], DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK]] 2023-08-15 16:01:22,696 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:22,696 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-15 16:01:22,699 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59652, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-15 16:01:22,702 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-15 16:01:22,702 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 16:01:22,704 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40443%2C1692115282123.meta, suffix=.meta, logDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/jenkins-hbase4.apache.org,40443,1692115282123, archiveDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/oldWALs, maxLogs=32 2023-08-15 16:01:22,719 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK] 2023-08-15 16:01:22,719 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK] 2023-08-15 16:01:22,721 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/jenkins-hbase4.apache.org,40443,1692115282123/jenkins-hbase4.apache.org%2C40443%2C1692115282123.meta.1692115282704.meta 2023-08-15 16:01:22,722 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK], DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK]] 2023-08-15 16:01:22,722 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-15 16:01:22,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-15 16:01:22,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-15 16:01:22,723 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-15 16:01:22,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-15 16:01:22,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 16:01:22,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-15 16:01:22,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-15 16:01:22,725 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-15 16:01:22,726 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/info 2023-08-15 16:01:22,726 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/info 2023-08-15 16:01:22,726 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-15 16:01:22,727 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:01:22,727 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-15 16:01:22,728 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/rep_barrier 2023-08-15 16:01:22,728 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/rep_barrier 2023-08-15 16:01:22,728 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-15 16:01:22,729 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:01:22,729 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-15 16:01:22,730 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/table 2023-08-15 16:01:22,730 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/table 2023-08-15 16:01:22,730 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-15 16:01:22,730 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:01:22,731 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740 2023-08-15 16:01:22,732 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740 2023-08-15 16:01:22,734 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-08-15 16:01:22,736 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-15 16:01:22,736 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=882084, jitterRate=0.12162788212299347}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-08-15 16:01:22,737 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-15 16:01:22,737 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692115282696 2023-08-15 16:01:22,741 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-15 16:01:22,742 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-15 16:01:22,742 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40443,1692115282123, state=OPEN 2023-08-15 16:01:22,743 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-15 16:01:22,743 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-15 16:01:22,745 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-15 16:01:22,745 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40443,1692115282123 in 201 msec 2023-08-15 16:01:22,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-15 16:01:22,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 363 msec 2023-08-15 16:01:22,748 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 440 msec 2023-08-15 16:01:22,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692115282748, completionTime=-1 2023-08-15 16:01:22,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-08-15 16:01:22,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-15 16:01:22,751 DEBUG [hconnection-0x31ea275d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-15 16:01:22,752 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59656, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-15 16:01:22,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-08-15 16:01:22,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692115342754 2023-08-15 16:01:22,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692115402754 2023-08-15 16:01:22,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-15 16:01:22,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39597,1692115282045-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39597,1692115282045-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39597,1692115282045-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39597, period=300000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-15 16:01:22,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-15 16:01:22,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-15 16:01:22,761 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-15 16:01:22,761 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-15 16:01:22,762 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-15 16:01:22,763 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-15 16:01:22,764 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/.tmp/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:22,764 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/.tmp/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605 empty. 2023-08-15 16:01:22,765 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/.tmp/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:22,765 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-15 16:01:23,177 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-15 16:01:23,178 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 98016bdbeaeb111a9e6a1aaf6184a605, NAME => 'hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/.tmp 2023-08-15 16:01:23,187 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 16:01:23,188 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 98016bdbeaeb111a9e6a1aaf6184a605, disabling compactions & flushes 2023-08-15 16:01:23,188 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,188 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,188 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. after waiting 0 ms 2023-08-15 16:01:23,188 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,188 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,188 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 98016bdbeaeb111a9e6a1aaf6184a605: 2023-08-15 16:01:23,190 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-15 16:01:23,192 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115283191"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692115283191"}]},"ts":"1692115283191"} 2023-08-15 16:01:23,193 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-15 16:01:23,194 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-15 16:01:23,194 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115283194"}]},"ts":"1692115283194"} 2023-08-15 16:01:23,195 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-15 16:01:23,206 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=98016bdbeaeb111a9e6a1aaf6184a605, ASSIGN}] 2023-08-15 16:01:23,208 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=98016bdbeaeb111a9e6a1aaf6184a605, ASSIGN 2023-08-15 16:01:23,208 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=98016bdbeaeb111a9e6a1aaf6184a605, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40443,1692115282123; forceNewPlan=false, retain=false 2023-08-15 16:01:23,360 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=98016bdbeaeb111a9e6a1aaf6184a605, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:23,360 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115283360"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692115283360"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692115283360"}]},"ts":"1692115283360"} 2023-08-15 16:01:23,362 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 98016bdbeaeb111a9e6a1aaf6184a605, server=jenkins-hbase4.apache.org,40443,1692115282123}] 2023-08-15 16:01:23,517 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,517 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 98016bdbeaeb111a9e6a1aaf6184a605, NAME => 'hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605.', STARTKEY => '', ENDKEY => ''} 2023-08-15 16:01:23,517 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,517 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-15 16:01:23,517 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,517 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,518 INFO [StoreOpener-98016bdbeaeb111a9e6a1aaf6184a605-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,520 DEBUG [StoreOpener-98016bdbeaeb111a9e6a1aaf6184a605-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605/info 2023-08-15 16:01:23,520 DEBUG [StoreOpener-98016bdbeaeb111a9e6a1aaf6184a605-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605/info 2023-08-15 16:01:23,520 INFO [StoreOpener-98016bdbeaeb111a9e6a1aaf6184a605-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 98016bdbeaeb111a9e6a1aaf6184a605 columnFamilyName info 2023-08-15 16:01:23,521 INFO [StoreOpener-98016bdbeaeb111a9e6a1aaf6184a605-1] regionserver.HStore(310): Store=98016bdbeaeb111a9e6a1aaf6184a605/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-15 16:01:23,521 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,522 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,525 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,526 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-15 16:01:23,527 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 98016bdbeaeb111a9e6a1aaf6184a605; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=877913, jitterRate=0.11632509529590607}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-15 16:01:23,527 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 98016bdbeaeb111a9e6a1aaf6184a605: 2023-08-15 16:01:23,528 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605., pid=6, masterSystemTime=1692115283513 2023-08-15 16:01:23,529 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,529 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,530 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=98016bdbeaeb111a9e6a1aaf6184a605, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:23,530 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692115283530"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692115283530"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692115283530"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692115283530"}]},"ts":"1692115283530"} 2023-08-15 16:01:23,532 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-15 16:01:23,533 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 98016bdbeaeb111a9e6a1aaf6184a605, server=jenkins-hbase4.apache.org,40443,1692115282123 in 169 msec 2023-08-15 16:01:23,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-15 16:01:23,535 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=98016bdbeaeb111a9e6a1aaf6184a605, ASSIGN in 327 msec 2023-08-15 16:01:23,535 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-15 16:01:23,536 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692115283535"}]},"ts":"1692115283535"} 2023-08-15 16:01:23,537 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-15 16:01:23,539 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-15 16:01:23,540 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 779 msec 2023-08-15 16:01:23,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-15 16:01:23,565 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-15 16:01:23,565 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:23,568 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-15 16:01:23,574 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-15 16:01:23,577 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 8 msec 2023-08-15 16:01:23,579 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-15 16:01:23,586 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-15 16:01:23,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 9 msec 2023-08-15 16:01:23,593 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-15 16:01:23,597 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-15 16:01:23,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.423sec 2023-08-15 16:01:23,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-15 16:01:23,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-15 16:01:23,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-15 16:01:23,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39597,1692115282045-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-15 16:01:23,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39597,1692115282045-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-15 16:01:23,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-15 16:01:23,661 DEBUG [Listener at localhost/41601] zookeeper.ReadOnlyZKClient(139): Connect 0x6f209331 to 127.0.0.1:51285 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-15 16:01:23,667 DEBUG [Listener at localhost/41601] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5cc1f58f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-15 16:01:23,669 DEBUG [hconnection-0x23ac3b4e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-15 16:01:23,671 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59662, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-15 16:01:23,672 INFO [Listener at localhost/41601] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:23,672 INFO [Listener at localhost/41601] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-15 16:01:23,675 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-15 16:01:23,675 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:23,676 INFO [Listener at localhost/41601] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-08-15 16:01:23,676 INFO [Listener at localhost/41601] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-15 16:01:23,678 INFO [Listener at localhost/41601] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=test.com%2C8080%2C1, suffix=, logDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/test.com,8080,1, archiveDir=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/oldWALs, maxLogs=32 2023-08-15 16:01:23,693 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK] 2023-08-15 16:01:23,694 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK] 2023-08-15 16:01:23,696 INFO [Listener at localhost/41601] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/test.com,8080,1/test.com%2C8080%2C1.1692115283679 2023-08-15 16:01:23,696 DEBUG [Listener at localhost/41601] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK], DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK]] 2023-08-15 16:01:23,709 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK] 2023-08-15 16:01:23,710 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK] 2023-08-15 16:01:23,711 INFO [Listener at localhost/41601] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/test.com,8080,1/test.com%2C8080%2C1.1692115283679 with entries=0, filesize=83 B; new WAL /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/test.com,8080,1/test.com%2C8080%2C1.1692115283696 2023-08-15 16:01:23,712 DEBUG [Listener at localhost/41601] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32867,DS-61e70227-eee4-4aba-81c2-455547626ad6,DISK], DatanodeInfoWithStorage[127.0.0.1:39463,DS-da9dd450-9207-4f8e-b7d8-cffd2d5c5a3f,DISK]] 2023-08-15 16:01:23,712 DEBUG [Listener at localhost/41601] wal.AbstractFSWAL(716): hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/test.com,8080,1/test.com%2C8080%2C1.1692115283679 is not closed yet, will try archiving it next time 2023-08-15 16:01:23,717 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/test.com,8080,1/test.com%2C8080%2C1.1692115283679 to hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/oldWALs/test.com%2C8080%2C1.1692115283679 2023-08-15 16:01:23,719 DEBUG [Listener at localhost/41601] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/oldWALs 2023-08-15 16:01:23,719 INFO [Listener at localhost/41601] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL test.com%2C8080%2C1:(num 1692115283696) 2023-08-15 16:01:23,719 INFO [Listener at localhost/41601] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-15 16:01:23,720 DEBUG [Listener at localhost/41601] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6f209331 to 127.0.0.1:51285 2023-08-15 16:01:23,720 DEBUG [Listener at localhost/41601] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:23,720 DEBUG [Listener at localhost/41601] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-15 16:01:23,720 DEBUG [Listener at localhost/41601] util.JVMClusterUtil(257): Found active master hash=1488729287, stopped=false 2023-08-15 16:01:23,720 INFO [Listener at localhost/41601] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:23,723 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-15 16:01:23,723 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-15 16:01:23,723 INFO [Listener at localhost/41601] procedure2.ProcedureExecutor(629): Stopping 2023-08-15 16:01:23,723 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:23,724 DEBUG [Listener at localhost/41601] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4e49cb05 to 127.0.0.1:51285 2023-08-15 16:01:23,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 16:01:23,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-15 16:01:23,724 DEBUG [Listener at localhost/41601] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:23,724 INFO [Listener at localhost/41601] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40443,1692115282123' ***** 2023-08-15 16:01:23,725 INFO [Listener at localhost/41601] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-15 16:01:23,725 INFO [RS:0;jenkins-hbase4:40443] regionserver.HeapMemoryManager(220): Stopping 2023-08-15 16:01:23,725 INFO [RS:0;jenkins-hbase4:40443] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-15 16:01:23,725 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-15 16:01:23,725 INFO [RS:0;jenkins-hbase4:40443] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-15 16:01:23,726 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(3305): Received CLOSE for 98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,726 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:23,726 DEBUG [RS:0;jenkins-hbase4:40443] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x04ac5d37 to 127.0.0.1:51285 2023-08-15 16:01:23,726 DEBUG [RS:0;jenkins-hbase4:40443] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:23,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 98016bdbeaeb111a9e6a1aaf6184a605, disabling compactions & flushes 2023-08-15 16:01:23,726 INFO [RS:0;jenkins-hbase4:40443] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-15 16:01:23,726 INFO [RS:0;jenkins-hbase4:40443] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-15 16:01:23,726 INFO [RS:0;jenkins-hbase4:40443] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-15 16:01:23,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,727 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-15 16:01:23,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. after waiting 0 ms 2023-08-15 16:01:23,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 98016bdbeaeb111a9e6a1aaf6184a605 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-15 16:01:23,727 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-08-15 16:01:23,727 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1478): Online Regions={98016bdbeaeb111a9e6a1aaf6184a605=hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605., 1588230740=hbase:meta,,1.1588230740} 2023-08-15 16:01:23,727 DEBUG [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1504): Waiting on 1588230740, 98016bdbeaeb111a9e6a1aaf6184a605 2023-08-15 16:01:23,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-15 16:01:23,733 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-15 16:01:23,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-15 16:01:23,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-15 16:01:23,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-15 16:01:23,733 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-08-15 16:01:23,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605/.tmp/info/23aee85476704b24aa3a19a813a43e9d 2023-08-15 16:01:23,760 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/.tmp/info/861688f01ab949128ca002da75e171bf 2023-08-15 16:01:23,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605/.tmp/info/23aee85476704b24aa3a19a813a43e9d as hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605/info/23aee85476704b24aa3a19a813a43e9d 2023-08-15 16:01:23,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605/info/23aee85476704b24aa3a19a813a43e9d, entries=2, sequenceid=6, filesize=4.8 K 2023-08-15 16:01:23,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 98016bdbeaeb111a9e6a1aaf6184a605 in 42ms, sequenceid=6, compaction requested=false 2023-08-15 16:01:23,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/namespace/98016bdbeaeb111a9e6a1aaf6184a605/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-15 16:01:23,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 98016bdbeaeb111a9e6a1aaf6184a605: 2023-08-15 16:01:23,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692115282760.98016bdbeaeb111a9e6a1aaf6184a605. 2023-08-15 16:01:23,785 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/.tmp/table/adcdc14a1f9b4a26a12794f9c60c6439 2023-08-15 16:01:23,791 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/.tmp/info/861688f01ab949128ca002da75e171bf as hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/info/861688f01ab949128ca002da75e171bf 2023-08-15 16:01:23,796 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/info/861688f01ab949128ca002da75e171bf, entries=10, sequenceid=9, filesize=5.9 K 2023-08-15 16:01:23,799 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/.tmp/table/adcdc14a1f9b4a26a12794f9c60c6439 as hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/table/adcdc14a1f9b4a26a12794f9c60c6439 2023-08-15 16:01:23,804 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/table/adcdc14a1f9b4a26a12794f9c60c6439, entries=2, sequenceid=9, filesize=4.7 K 2023-08-15 16:01:23,805 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 72ms, sequenceid=9, compaction requested=false 2023-08-15 16:01:23,811 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-08-15 16:01:23,812 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-15 16:01:23,812 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-15 16:01:23,812 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-15 16:01:23,812 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-15 16:01:23,927 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40443,1692115282123; all regions closed. 2023-08-15 16:01:23,930 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/WALs/jenkins-hbase4.apache.org,40443,1692115282123/jenkins-hbase4.apache.org%2C40443%2C1692115282123.meta.1692115282704.meta not finished, retry = 0 2023-08-15 16:01:24,033 DEBUG [RS:0;jenkins-hbase4:40443] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/oldWALs 2023-08-15 16:01:24,033 INFO [RS:0;jenkins-hbase4:40443] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40443%2C1692115282123.meta:.meta(num 1692115282704) 2023-08-15 16:01:24,038 DEBUG [RS:0;jenkins-hbase4:40443] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/oldWALs 2023-08-15 16:01:24,038 INFO [RS:0;jenkins-hbase4:40443] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40443%2C1692115282123:(num 1692115282548) 2023-08-15 16:01:24,038 DEBUG [RS:0;jenkins-hbase4:40443] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:24,038 INFO [RS:0;jenkins-hbase4:40443] regionserver.LeaseManager(133): Closed leases 2023-08-15 16:01:24,038 INFO [RS:0;jenkins-hbase4:40443] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-15 16:01:24,038 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-15 16:01:24,039 INFO [RS:0;jenkins-hbase4:40443] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40443 2023-08-15 16:01:24,043 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 16:01:24,043 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40443,1692115282123 2023-08-15 16:01:24,043 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-15 16:01:24,045 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40443,1692115282123] 2023-08-15 16:01:24,045 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40443,1692115282123; numProcessing=1 2023-08-15 16:01:24,046 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40443,1692115282123 already deleted, retry=false 2023-08-15 16:01:24,046 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40443,1692115282123 expired; onlineServers=0 2023-08-15 16:01:24,046 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39597,1692115282045' ***** 2023-08-15 16:01:24,046 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-15 16:01:24,046 DEBUG [M:0;jenkins-hbase4:39597] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c83bb08, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-15 16:01:24,046 INFO [M:0;jenkins-hbase4:39597] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39597,1692115282045 2023-08-15 16:01:24,046 INFO [M:0;jenkins-hbase4:39597] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39597,1692115282045; all regions closed. 2023-08-15 16:01:24,046 DEBUG [M:0;jenkins-hbase4:39597] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-15 16:01:24,046 DEBUG [M:0;jenkins-hbase4:39597] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-15 16:01:24,046 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-15 16:01:24,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115282317] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692115282317,5,FailOnTimeoutGroup] 2023-08-15 16:01:24,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115282318] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692115282318,5,FailOnTimeoutGroup] 2023-08-15 16:01:24,046 DEBUG [M:0;jenkins-hbase4:39597] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-15 16:01:24,047 INFO [M:0;jenkins-hbase4:39597] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-15 16:01:24,047 INFO [M:0;jenkins-hbase4:39597] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-15 16:01:24,047 INFO [M:0;jenkins-hbase4:39597] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-15 16:01:24,048 DEBUG [M:0;jenkins-hbase4:39597] master.HMaster(1512): Stopping service threads 2023-08-15 16:01:24,048 INFO [M:0;jenkins-hbase4:39597] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-15 16:01:24,048 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-15 16:01:24,048 ERROR [M:0;jenkins-hbase4:39597] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] 2023-08-15 16:01:24,048 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-15 16:01:24,048 INFO [M:0;jenkins-hbase4:39597] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-15 16:01:24,048 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-15 16:01:24,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-15 16:01:24,049 DEBUG [M:0;jenkins-hbase4:39597] zookeeper.ZKUtil(398): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-15 16:01:24,049 WARN [M:0;jenkins-hbase4:39597] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-15 16:01:24,049 INFO [M:0;jenkins-hbase4:39597] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-15 16:01:24,049 INFO [M:0;jenkins-hbase4:39597] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-15 16:01:24,049 DEBUG [M:0;jenkins-hbase4:39597] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-15 16:01:24,049 INFO [M:0;jenkins-hbase4:39597] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:24,049 DEBUG [M:0;jenkins-hbase4:39597] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:24,049 DEBUG [M:0;jenkins-hbase4:39597] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-15 16:01:24,049 DEBUG [M:0;jenkins-hbase4:39597] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:24,050 INFO [M:0;jenkins-hbase4:39597] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-08-15 16:01:24,059 INFO [M:0;jenkins-hbase4:39597] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e08bd4e9d7d04f8f881e692d2ebed88a 2023-08-15 16:01:24,064 DEBUG [M:0;jenkins-hbase4:39597] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e08bd4e9d7d04f8f881e692d2ebed88a as hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e08bd4e9d7d04f8f881e692d2ebed88a 2023-08-15 16:01:24,068 INFO [M:0;jenkins-hbase4:39597] regionserver.HStore(1080): Added hdfs://localhost:43307/user/jenkins/test-data/da95fc9f-f90e-636d-ca82-f9bdfeb249ba/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e08bd4e9d7d04f8f881e692d2ebed88a, entries=8, sequenceid=66, filesize=6.3 K 2023-08-15 16:01:24,069 INFO [M:0;jenkins-hbase4:39597] regionserver.HRegion(2948): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=66, compaction requested=false 2023-08-15 16:01:24,070 INFO [M:0;jenkins-hbase4:39597] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-15 16:01:24,070 DEBUG [M:0;jenkins-hbase4:39597] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-15 16:01:24,073 INFO [M:0;jenkins-hbase4:39597] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-15 16:01:24,073 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-15 16:01:24,073 INFO [M:0;jenkins-hbase4:39597] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39597 2023-08-15 16:01:24,076 DEBUG [M:0;jenkins-hbase4:39597] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39597,1692115282045 already deleted, retry=false 2023-08-15 16:01:24,145 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 16:01:24,145 INFO [RS:0;jenkins-hbase4:40443] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40443,1692115282123; zookeeper connection closed. 2023-08-15 16:01:24,145 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): regionserver:40443-0x10010ad600d0001, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 16:01:24,146 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4d5c494e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4d5c494e 2023-08-15 16:01:24,146 INFO [Listener at localhost/41601] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-08-15 16:01:24,245 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 16:01:24,245 INFO [M:0;jenkins-hbase4:39597] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39597,1692115282045; zookeeper connection closed. 2023-08-15 16:01:24,245 DEBUG [Listener at localhost/41601-EventThread] zookeeper.ZKWatcher(600): master:39597-0x10010ad600d0000, quorum=127.0.0.1:51285, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-15 16:01:24,247 WARN [Listener at localhost/41601] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 16:01:24,250 INFO [Listener at localhost/41601] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 16:01:24,253 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-15 16:01:24,353 WARN [BP-272164748-172.31.14.131-1692115281352 heartbeating to localhost/127.0.0.1:43307] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 16:01:24,354 WARN [BP-272164748-172.31.14.131-1692115281352 heartbeating to localhost/127.0.0.1:43307] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-272164748-172.31.14.131-1692115281352 (Datanode Uuid 5bf794f4-e12b-4608-966f-fba7b4b7519a) service to localhost/127.0.0.1:43307 2023-08-15 16:01:24,354 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/cluster_eb197253-fe0d-4177-c180-83b80fdd7673/dfs/data/data3/current/BP-272164748-172.31.14.131-1692115281352] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 16:01:24,355 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/cluster_eb197253-fe0d-4177-c180-83b80fdd7673/dfs/data/data4/current/BP-272164748-172.31.14.131-1692115281352] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 16:01:24,355 WARN [Listener at localhost/41601] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-15 16:01:24,359 INFO [Listener at localhost/41601] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 16:01:24,417 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-15 16:01:24,462 WARN [BP-272164748-172.31.14.131-1692115281352 heartbeating to localhost/127.0.0.1:43307] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-15 16:01:24,462 WARN [BP-272164748-172.31.14.131-1692115281352 heartbeating to localhost/127.0.0.1:43307] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-272164748-172.31.14.131-1692115281352 (Datanode Uuid 746804e0-8861-4494-a303-587f29c10b9d) service to localhost/127.0.0.1:43307 2023-08-15 16:01:24,463 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/cluster_eb197253-fe0d-4177-c180-83b80fdd7673/dfs/data/data1/current/BP-272164748-172.31.14.131-1692115281352] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 16:01:24,463 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3add03e2-1a34-241c-d053-a37975dec82d/cluster_eb197253-fe0d-4177-c180-83b80fdd7673/dfs/data/data2/current/BP-272164748-172.31.14.131-1692115281352] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-15 16:01:24,474 INFO [Listener at localhost/41601] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-15 16:01:24,585 INFO [Listener at localhost/41601] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-15 16:01:24,596 INFO [Listener at localhost/41601] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-08-15 16:01:24,606 INFO [Listener at localhost/41601] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=99 (was 77) - Thread LEAK? -, OpenFileDescriptor=488 (was 471) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=143 (was 147), ProcessCount=169 (was 169), AvailableMemoryMB=625 (was 771)