2023-09-04 07:58:03,599 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6 2023-09-04 07:58:03,613 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling timeout: 13 mins 2023-09-04 07:58:03,651 INFO [Time-limited test] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=214, ProcessCount=167, AvailableMemoryMB=9722 2023-09-04 07:58:03,657 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-04 07:58:03,658 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb, deleteOnExit=true 2023-09-04 07:58:03,658 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-04 07:58:03,659 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/test.cache.data in system properties and HBase conf 2023-09-04 07:58:03,659 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/hadoop.tmp.dir in system properties and HBase conf 2023-09-04 07:58:03,660 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/hadoop.log.dir in system properties and HBase conf 2023-09-04 07:58:03,660 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-04 07:58:03,661 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-04 07:58:03,661 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-04 07:58:03,768 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-09-04 07:58:04,144 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-04 07:58:04,149 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-04 07:58:04,149 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-04 07:58:04,150 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-04 07:58:04,150 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 07:58:04,150 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-04 07:58:04,151 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-04 07:58:04,151 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 07:58:04,152 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 07:58:04,152 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-04 07:58:04,152 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/nfs.dump.dir in system properties and HBase conf 2023-09-04 07:58:04,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/java.io.tmpdir in system properties and HBase conf 2023-09-04 07:58:04,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 07:58:04,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-04 07:58:04,154 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-04 07:58:04,681 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 07:58:04,685 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 07:58:04,992 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-09-04 07:58:05,164 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-09-04 07:58:05,184 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:05,224 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:05,283 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/java.io.tmpdir/Jetty_localhost_46503_hdfs____w6s5wg/webapp 2023-09-04 07:58:05,423 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46503 2023-09-04 07:58:05,434 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 07:58:05,434 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 07:58:05,925 WARN [Listener at localhost/39519] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:05,999 WARN [Listener at localhost/39519] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:58:06,017 WARN [Listener at localhost/39519] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:06,023 INFO [Listener at localhost/39519] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:06,027 INFO [Listener at localhost/39519] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/java.io.tmpdir/Jetty_localhost_43579_datanode____a9sstb/webapp 2023-09-04 07:58:06,122 INFO [Listener at localhost/39519] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43579 2023-09-04 07:58:06,427 WARN [Listener at localhost/44759] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:06,444 WARN [Listener at localhost/44759] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:58:06,449 WARN [Listener at localhost/44759] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:06,452 INFO [Listener at localhost/44759] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:06,458 INFO [Listener at localhost/44759] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/java.io.tmpdir/Jetty_localhost_34091_datanode____iy1c4u/webapp 2023-09-04 07:58:06,566 INFO [Listener at localhost/44759] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34091 2023-09-04 07:58:06,577 WARN [Listener at localhost/35195] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:06,976 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6b96a6a87698e32: Processing first storage report for DS-ac57ca98-79e4-40e5-a55a-812fa81554a4 from datanode 7d45ad25-d115-483f-b028-6ae759d392aa 2023-09-04 07:58:06,978 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6b96a6a87698e32: from storage DS-ac57ca98-79e4-40e5-a55a-812fa81554a4 node DatanodeRegistration(127.0.0.1:34053, datanodeUuid=7d45ad25-d115-483f-b028-6ae759d392aa, infoPort=46439, infoSecurePort=0, ipcPort=44759, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: true, processing time: 3 msecs, invalidatedBlocks: 0 2023-09-04 07:58:06,979 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xab13e020f9d54fa7: Processing first storage report for DS-00499317-c545-4ad0-a7e8-abab41c215db from datanode 71fc19a9-265d-4ab6-b211-2890fd2d7ee2 2023-09-04 07:58:06,979 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xab13e020f9d54fa7: from storage DS-00499317-c545-4ad0-a7e8-abab41c215db node DatanodeRegistration(127.0.0.1:39997, datanodeUuid=71fc19a9-265d-4ab6-b211-2890fd2d7ee2, infoPort=33643, infoSecurePort=0, ipcPort=35195, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:06,979 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6b96a6a87698e32: Processing first storage report for DS-bd10e2cd-96cf-4943-b6bf-4f6634ef4805 from datanode 7d45ad25-d115-483f-b028-6ae759d392aa 2023-09-04 07:58:06,979 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6b96a6a87698e32: from storage DS-bd10e2cd-96cf-4943-b6bf-4f6634ef4805 node DatanodeRegistration(127.0.0.1:34053, datanodeUuid=7d45ad25-d115-483f-b028-6ae759d392aa, infoPort=46439, infoSecurePort=0, ipcPort=44759, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-04 07:58:06,979 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xab13e020f9d54fa7: Processing first storage report for DS-976c5982-d31b-4d41-ae91-c3d57155497a from datanode 71fc19a9-265d-4ab6-b211-2890fd2d7ee2 2023-09-04 07:58:06,980 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xab13e020f9d54fa7: from storage DS-976c5982-d31b-4d41-ae91-c3d57155497a node DatanodeRegistration(127.0.0.1:39997, datanodeUuid=71fc19a9-265d-4ab6-b211-2890fd2d7ee2, infoPort=33643, infoSecurePort=0, ipcPort=35195, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:07,057 DEBUG [Listener at localhost/35195] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6 2023-09-04 07:58:07,130 INFO [Listener at localhost/35195] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/zookeeper_0, clientPort=58270, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-04 07:58:07,148 INFO [Listener at localhost/35195] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=58270 2023-09-04 07:58:07,159 INFO [Listener at localhost/35195] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:07,162 INFO [Listener at localhost/35195] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:07,839 INFO [Listener at localhost/35195] util.FSUtils(471): Created version file at hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb with version=8 2023-09-04 07:58:07,840 INFO [Listener at localhost/35195] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/hbase-staging 2023-09-04 07:58:08,151 INFO [Listener at localhost/35195] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-09-04 07:58:08,660 INFO [Listener at localhost/35195] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 07:58:08,693 INFO [Listener at localhost/35195] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:08,694 INFO [Listener at localhost/35195] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:08,694 INFO [Listener at localhost/35195] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 07:58:08,694 INFO [Listener at localhost/35195] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:08,695 INFO [Listener at localhost/35195] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 07:58:08,843 INFO [Listener at localhost/35195] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 07:58:08,914 DEBUG [Listener at localhost/35195] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-09-04 07:58:09,007 INFO [Listener at localhost/35195] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33813 2023-09-04 07:58:09,018 INFO [Listener at localhost/35195] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:09,020 INFO [Listener at localhost/35195] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:09,040 INFO [Listener at localhost/35195] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33813 connecting to ZooKeeper ensemble=127.0.0.1:58270 2023-09-04 07:58:09,079 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:338130x0, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 07:58:09,082 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33813-0x10075f222ab0000 connected 2023-09-04 07:58:09,105 DEBUG [Listener at localhost/35195] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 07:58:09,106 DEBUG [Listener at localhost/35195] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:58:09,110 DEBUG [Listener at localhost/35195] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 07:58:09,117 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33813 2023-09-04 07:58:09,118 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33813 2023-09-04 07:58:09,118 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33813 2023-09-04 07:58:09,119 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33813 2023-09-04 07:58:09,119 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33813 2023-09-04 07:58:09,126 INFO [Listener at localhost/35195] master.HMaster(444): hbase.rootdir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb, hbase.cluster.distributed=false 2023-09-04 07:58:09,191 INFO [Listener at localhost/35195] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 07:58:09,191 INFO [Listener at localhost/35195] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:09,192 INFO [Listener at localhost/35195] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:09,192 INFO [Listener at localhost/35195] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 07:58:09,192 INFO [Listener at localhost/35195] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:09,192 INFO [Listener at localhost/35195] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 07:58:09,197 INFO [Listener at localhost/35195] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 07:58:09,200 INFO [Listener at localhost/35195] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42995 2023-09-04 07:58:09,202 INFO [Listener at localhost/35195] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-04 07:58:09,210 DEBUG [Listener at localhost/35195] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-04 07:58:09,211 INFO [Listener at localhost/35195] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:09,214 INFO [Listener at localhost/35195] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:09,216 INFO [Listener at localhost/35195] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42995 connecting to ZooKeeper ensemble=127.0.0.1:58270 2023-09-04 07:58:09,220 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): regionserver:429950x0, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 07:58:09,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42995-0x10075f222ab0001 connected 2023-09-04 07:58:09,221 DEBUG [Listener at localhost/35195] zookeeper.ZKUtil(164): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 07:58:09,222 DEBUG [Listener at localhost/35195] zookeeper.ZKUtil(164): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:58:09,223 DEBUG [Listener at localhost/35195] zookeeper.ZKUtil(164): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 07:58:09,224 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42995 2023-09-04 07:58:09,224 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42995 2023-09-04 07:58:09,225 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42995 2023-09-04 07:58:09,226 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42995 2023-09-04 07:58:09,226 DEBUG [Listener at localhost/35195] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42995 2023-09-04 07:58:09,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:09,243 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 07:58:09,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:09,264 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 07:58:09,264 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 07:58:09,264 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:09,265 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 07:58:09,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33813,1693814287978 from backup master directory 2023-09-04 07:58:09,266 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 07:58:09,269 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:09,269 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 07:58:09,270 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 07:58:09,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:09,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-09-04 07:58:09,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-09-04 07:58:09,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/hbase.id with ID: e3ac2c54-ddae-49d9-99a8-f0b001a79473 2023-09-04 07:58:09,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:09,410 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:09,453 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x023effef to 127.0.0.1:58270 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:58:09,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ce8f404, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:58:09,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 07:58:09,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-04 07:58:09,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:58:09,563 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store-tmp 2023-09-04 07:58:09,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:09,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 07:58:09,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:09,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:09,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 07:58:09,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:09,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:09,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 07:58:09,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:09,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33813%2C1693814287978, suffix=, logDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978, archiveDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/oldWALs, maxLogs=10 2023-09-04 07:58:09,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-09-04 07:58:09,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-09-04 07:58:09,653 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:190) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:160) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200) at org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:220) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:348) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:09,687 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:417) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:09,690 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34053,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK] 2023-09-04 07:58:09,690 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39997,DS-00499317-c545-4ad0-a7e8-abab41c215db,DISK] 2023-09-04 07:58:09,699 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:09,763 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 2023-09-04 07:58:09,763 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34053,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK], DatanodeInfoWithStorage[127.0.0.1:39997,DS-00499317-c545-4ad0-a7e8-abab41c215db,DISK]] 2023-09-04 07:58:09,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:58:09,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:09,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:09,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:09,823 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:09,830 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-04 07:58:09,857 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-04 07:58:09,870 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:09,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:09,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:09,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:09,898 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:58:09,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=702973, jitterRate=-0.10612477362155914}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:58:09,899 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 07:58:09,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-04 07:58:09,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-04 07:58:09,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-04 07:58:09,923 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-04 07:58:09,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-09-04 07:58:09,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 31 msec 2023-09-04 07:58:09,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-04 07:58:09,981 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-04 07:58:09,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-04 07:58:10,010 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-04 07:58:10,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-04 07:58:10,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-04 07:58:10,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-04 07:58:10,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-04 07:58:10,031 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:10,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-04 07:58:10,033 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-04 07:58:10,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-04 07:58:10,051 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 07:58:10,051 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 07:58:10,051 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:10,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33813,1693814287978, sessionid=0x10075f222ab0000, setting cluster-up flag (Was=false) 2023-09-04 07:58:10,068 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:10,073 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-04 07:58:10,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:10,081 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:10,089 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-04 07:58:10,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:10,093 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.hbase-snapshot/.tmp 2023-09-04 07:58:10,130 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(951): ClusterId : e3ac2c54-ddae-49d9-99a8-f0b001a79473 2023-09-04 07:58:10,135 DEBUG [RS:0;jenkins-hbase4:42995] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-04 07:58:10,140 DEBUG [RS:0;jenkins-hbase4:42995] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-04 07:58:10,140 DEBUG [RS:0;jenkins-hbase4:42995] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-04 07:58:10,143 DEBUG [RS:0;jenkins-hbase4:42995] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-04 07:58:10,144 DEBUG [RS:0;jenkins-hbase4:42995] zookeeper.ReadOnlyZKClient(139): Connect 0x67e7c1db to 127.0.0.1:58270 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:58:10,150 DEBUG [RS:0;jenkins-hbase4:42995] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69202357, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:58:10,152 DEBUG [RS:0;jenkins-hbase4:42995] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6d1faaab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 07:58:10,185 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42995 2023-09-04 07:58:10,190 INFO [RS:0;jenkins-hbase4:42995] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-04 07:58:10,190 INFO [RS:0;jenkins-hbase4:42995] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-04 07:58:10,190 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1022): About to register with Master. 2023-09-04 07:58:10,194 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33813,1693814287978 with isa=jenkins-hbase4.apache.org/172.31.14.131:42995, startcode=1693814289190 2023-09-04 07:58:10,216 DEBUG [RS:0;jenkins-hbase4:42995] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-04 07:58:10,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-04 07:58:10,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:58:10,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:58:10,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:58:10,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:58:10,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-04 07:58:10,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 07:58:10,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693814320238 2023-09-04 07:58:10,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-04 07:58:10,254 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-04 07:58:10,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-04 07:58:10,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-04 07:58:10,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-04 07:58:10,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-04 07:58:10,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,263 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 07:58:10,264 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-04 07:58:10,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-04 07:58:10,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-04 07:58:10,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-04 07:58:10,270 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 07:58:10,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-04 07:58:10,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-04 07:58:10,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814290276,5,FailOnTimeoutGroup] 2023-09-04 07:58:10,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814290281,5,FailOnTimeoutGroup] 2023-09-04 07:58:10,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-04 07:58:10,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,290 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42113, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-09-04 07:58:10,305 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33813] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:10,319 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 07:58:10,320 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 07:58:10,321 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb 2023-09-04 07:58:10,330 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb 2023-09-04 07:58:10,330 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39519 2023-09-04 07:58:10,330 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-04 07:58:10,340 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 07:58:10,341 DEBUG [RS:0;jenkins-hbase4:42995] zookeeper.ZKUtil(162): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:10,341 WARN [RS:0;jenkins-hbase4:42995] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 07:58:10,345 INFO [RS:0;jenkins-hbase4:42995] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:58:10,343 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42995,1693814289190] 2023-09-04 07:58:10,345 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:10,356 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:10,357 DEBUG [RS:0;jenkins-hbase4:42995] zookeeper.ZKUtil(162): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:10,359 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 07:58:10,362 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/info 2023-09-04 07:58:10,363 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 07:58:10,366 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:10,366 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 07:58:10,369 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:58:10,370 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 07:58:10,370 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-04 07:58:10,371 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:10,372 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 07:58:10,376 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/table 2023-09-04 07:58:10,377 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 07:58:10,378 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:10,382 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740 2023-09-04 07:58:10,384 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740 2023-09-04 07:58:10,384 INFO [RS:0;jenkins-hbase4:42995] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-04 07:58:10,388 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 07:58:10,391 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 07:58:10,395 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:58:10,396 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=748517, jitterRate=-0.04821181297302246}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 07:58:10,396 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 07:58:10,396 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 07:58:10,396 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 07:58:10,396 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 07:58:10,396 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 07:58:10,396 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 07:58:10,397 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 07:58:10,398 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 07:58:10,408 INFO [RS:0;jenkins-hbase4:42995] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-04 07:58:10,409 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 07:58:10,409 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-04 07:58:10,413 INFO [RS:0;jenkins-hbase4:42995] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 07:58:10,413 INFO [RS:0;jenkins-hbase4:42995] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,417 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-04 07:58:10,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-04 07:58:10,427 INFO [RS:0;jenkins-hbase4:42995] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,427 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,428 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,428 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,428 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,429 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,429 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 07:58:10,429 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,429 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,429 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,429 DEBUG [RS:0;jenkins-hbase4:42995] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:10,432 INFO [RS:0;jenkins-hbase4:42995] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,432 INFO [RS:0;jenkins-hbase4:42995] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,432 INFO [RS:0;jenkins-hbase4:42995] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,436 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-04 07:58:10,439 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-04 07:58:10,450 INFO [RS:0;jenkins-hbase4:42995] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-04 07:58:10,452 INFO [RS:0;jenkins-hbase4:42995] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42995,1693814289190-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:10,468 INFO [RS:0;jenkins-hbase4:42995] regionserver.Replication(203): jenkins-hbase4.apache.org,42995,1693814289190 started 2023-09-04 07:58:10,468 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42995,1693814289190, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42995, sessionid=0x10075f222ab0001 2023-09-04 07:58:10,468 DEBUG [RS:0;jenkins-hbase4:42995] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-04 07:58:10,468 DEBUG [RS:0;jenkins-hbase4:42995] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:10,468 DEBUG [RS:0;jenkins-hbase4:42995] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42995,1693814289190' 2023-09-04 07:58:10,468 DEBUG [RS:0;jenkins-hbase4:42995] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:58:10,469 DEBUG [RS:0;jenkins-hbase4:42995] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:58:10,470 DEBUG [RS:0;jenkins-hbase4:42995] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-04 07:58:10,470 DEBUG [RS:0;jenkins-hbase4:42995] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-04 07:58:10,470 DEBUG [RS:0;jenkins-hbase4:42995] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:10,470 DEBUG [RS:0;jenkins-hbase4:42995] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42995,1693814289190' 2023-09-04 07:58:10,470 DEBUG [RS:0;jenkins-hbase4:42995] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-04 07:58:10,470 DEBUG [RS:0;jenkins-hbase4:42995] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-04 07:58:10,471 DEBUG [RS:0;jenkins-hbase4:42995] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-04 07:58:10,471 INFO [RS:0;jenkins-hbase4:42995] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-04 07:58:10,471 INFO [RS:0;jenkins-hbase4:42995] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-04 07:58:10,581 INFO [RS:0;jenkins-hbase4:42995] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42995%2C1693814289190, suffix=, logDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190, archiveDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/oldWALs, maxLogs=32 2023-09-04 07:58:10,591 DEBUG [jenkins-hbase4:33813] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-04 07:58:10,594 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42995,1693814289190, state=OPENING 2023-09-04 07:58:10,603 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34053,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK] 2023-09-04 07:58:10,603 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39997,DS-00499317-c545-4ad0-a7e8-abab41c215db,DISK] 2023-09-04 07:58:10,604 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-04 07:58:10,605 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:10,606 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 07:58:10,608 INFO [RS:0;jenkins-hbase4:42995] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 2023-09-04 07:58:10,609 DEBUG [RS:0;jenkins-hbase4:42995] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34053,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK], DatanodeInfoWithStorage[127.0.0.1:39997,DS-00499317-c545-4ad0-a7e8-abab41c215db,DISK]] 2023-09-04 07:58:10,611 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42995,1693814289190}] 2023-09-04 07:58:10,795 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:10,797 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-04 07:58:10,800 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55072, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-04 07:58:10,811 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-04 07:58:10,812 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:58:10,815 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta, suffix=.meta, logDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190, archiveDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/oldWALs, maxLogs=32 2023-09-04 07:58:10,832 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39997,DS-00499317-c545-4ad0-a7e8-abab41c215db,DISK] 2023-09-04 07:58:10,832 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34053,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK] 2023-09-04 07:58:10,838 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta 2023-09-04 07:58:10,838 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39997,DS-00499317-c545-4ad0-a7e8-abab41c215db,DISK], DatanodeInfoWithStorage[127.0.0.1:34053,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK]] 2023-09-04 07:58:10,839 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:58:10,840 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-04 07:58:10,857 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-04 07:58:10,862 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-04 07:58:10,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-04 07:58:10,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:10,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-04 07:58:10,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-04 07:58:10,871 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 07:58:10,873 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/info 2023-09-04 07:58:10,873 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/info 2023-09-04 07:58:10,874 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 07:58:10,874 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:10,875 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 07:58:10,876 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:58:10,876 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:58:10,877 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 07:58:10,877 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:10,878 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 07:58:10,879 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/table 2023-09-04 07:58:10,879 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/table 2023-09-04 07:58:10,880 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 07:58:10,880 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:10,882 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740 2023-09-04 07:58:10,885 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740 2023-09-04 07:58:10,888 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 07:58:10,890 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 07:58:10,891 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=814699, jitterRate=0.03594405949115753}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 07:58:10,891 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 07:58:10,901 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693814290788 2023-09-04 07:58:10,917 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-04 07:58:10,918 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-04 07:58:10,919 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42995,1693814289190, state=OPEN 2023-09-04 07:58:10,921 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-04 07:58:10,922 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 07:58:10,926 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-04 07:58:10,926 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42995,1693814289190 in 312 msec 2023-09-04 07:58:10,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-04 07:58:10,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 508 msec 2023-09-04 07:58:10,936 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 796 msec 2023-09-04 07:58:10,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693814290936, completionTime=-1 2023-09-04 07:58:10,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-04 07:58:10,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-04 07:58:11,001 DEBUG [hconnection-0x736e7d1b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 07:58:11,004 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55086, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 07:58:11,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-04 07:58:11,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693814351020 2023-09-04 07:58:11,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693814411020 2023-09-04 07:58:11,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 83 msec 2023-09-04 07:58:11,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33813,1693814287978-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:11,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33813,1693814287978-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:11,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33813,1693814287978-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:11,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33813, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:11,045 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:11,051 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-04 07:58:11,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-04 07:58:11,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 07:58:11,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-04 07:58:11,072 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 07:58:11,075 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 07:58:11,099 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,101 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a empty. 2023-09-04 07:58:11,102 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,102 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-04 07:58:11,131 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-04 07:58:11,133 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6b6bea4f63cc581dcd842e858450946a, NAME => 'hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp 2023-09-04 07:58:11,150 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:11,151 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 6b6bea4f63cc581dcd842e858450946a, disabling compactions & flushes 2023-09-04 07:58:11,151 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:11,151 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:11,151 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. after waiting 0 ms 2023-09-04 07:58:11,151 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:11,151 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:11,151 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 6b6bea4f63cc581dcd842e858450946a: 2023-09-04 07:58:11,155 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 07:58:11,171 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814291159"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814291159"}]},"ts":"1693814291159"} 2023-09-04 07:58:11,198 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 07:58:11,200 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 07:58:11,204 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814291200"}]},"ts":"1693814291200"} 2023-09-04 07:58:11,208 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-04 07:58:11,222 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6b6bea4f63cc581dcd842e858450946a, ASSIGN}] 2023-09-04 07:58:11,225 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6b6bea4f63cc581dcd842e858450946a, ASSIGN 2023-09-04 07:58:11,227 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6b6bea4f63cc581dcd842e858450946a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42995,1693814289190; forceNewPlan=false, retain=false 2023-09-04 07:58:11,378 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6b6bea4f63cc581dcd842e858450946a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:11,378 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814291377"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814291377"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814291377"}]},"ts":"1693814291377"} 2023-09-04 07:58:11,384 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 6b6bea4f63cc581dcd842e858450946a, server=jenkins-hbase4.apache.org,42995,1693814289190}] 2023-09-04 07:58:11,547 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:11,548 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6b6bea4f63cc581dcd842e858450946a, NAME => 'hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:58:11,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:11,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,552 INFO [StoreOpener-6b6bea4f63cc581dcd842e858450946a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,554 DEBUG [StoreOpener-6b6bea4f63cc581dcd842e858450946a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a/info 2023-09-04 07:58:11,554 DEBUG [StoreOpener-6b6bea4f63cc581dcd842e858450946a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a/info 2023-09-04 07:58:11,555 INFO [StoreOpener-6b6bea4f63cc581dcd842e858450946a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b6bea4f63cc581dcd842e858450946a columnFamilyName info 2023-09-04 07:58:11,556 INFO [StoreOpener-6b6bea4f63cc581dcd842e858450946a-1] regionserver.HStore(310): Store=6b6bea4f63cc581dcd842e858450946a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:11,557 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,558 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,563 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:11,566 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:58:11,567 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6b6bea4f63cc581dcd842e858450946a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=764947, jitterRate=-0.0273200124502182}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:58:11,567 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6b6bea4f63cc581dcd842e858450946a: 2023-09-04 07:58:11,570 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a., pid=6, masterSystemTime=1693814291538 2023-09-04 07:58:11,574 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:11,574 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:11,576 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6b6bea4f63cc581dcd842e858450946a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:11,577 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814291575"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814291575"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814291575"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814291575"}]},"ts":"1693814291575"} 2023-09-04 07:58:11,584 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-04 07:58:11,585 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 6b6bea4f63cc581dcd842e858450946a, server=jenkins-hbase4.apache.org,42995,1693814289190 in 196 msec 2023-09-04 07:58:11,589 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-04 07:58:11,589 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6b6bea4f63cc581dcd842e858450946a, ASSIGN in 362 msec 2023-09-04 07:58:11,591 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 07:58:11,591 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814291591"}]},"ts":"1693814291591"} 2023-09-04 07:58:11,595 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-04 07:58:11,599 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 07:58:11,602 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 538 msec 2023-09-04 07:58:11,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-04 07:58:11,674 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:58:11,674 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:11,714 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-04 07:58:11,730 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:58:11,736 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 33 msec 2023-09-04 07:58:11,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-04 07:58:11,761 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:58:11,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-09-04 07:58:11,776 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-04 07:58:11,779 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-04 07:58:11,779 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.509sec 2023-09-04 07:58:11,781 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-04 07:58:11,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-04 07:58:11,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-04 07:58:11,784 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33813,1693814287978-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-04 07:58:11,784 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33813,1693814287978-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-04 07:58:11,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-04 07:58:11,836 DEBUG [Listener at localhost/35195] zookeeper.ReadOnlyZKClient(139): Connect 0x42bfd752 to 127.0.0.1:58270 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:58:11,841 DEBUG [Listener at localhost/35195] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@156a897f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:58:11,875 DEBUG [hconnection-0x6b9564bb-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 07:58:11,887 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55100, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 07:58:11,898 INFO [Listener at localhost/35195] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:11,898 INFO [Listener at localhost/35195] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:11,907 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-04 07:58:11,907 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:11,908 INFO [Listener at localhost/35195] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-04 07:58:11,935 WARN [Listener at localhost/35195] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:58:11,939 WARN [Listener at localhost/35195] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:11,941 INFO [Listener at localhost/35195] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:11,949 INFO [Listener at localhost/35195] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/java.io.tmpdir/Jetty_localhost_39645_datanode____6nlu2g/webapp 2023-09-04 07:58:12,054 INFO [Listener at localhost/35195] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39645 2023-09-04 07:58:12,065 WARN [Listener at localhost/38313] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:12,101 WARN [Listener at localhost/38313] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:58:12,105 WARN [Listener at localhost/38313] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:12,107 INFO [Listener at localhost/38313] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:12,119 INFO [Listener at localhost/38313] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/java.io.tmpdir/Jetty_localhost_40547_datanode____.n2s2l/webapp 2023-09-04 07:58:12,197 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5fc5711379d2eafa: Processing first storage report for DS-0d033ba1-87bf-4942-8573-cd8f66247aff from datanode 34ec2e2b-8100-4f9f-8f8d-fa733114e19c 2023-09-04 07:58:12,197 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5fc5711379d2eafa: from storage DS-0d033ba1-87bf-4942-8573-cd8f66247aff node DatanodeRegistration(127.0.0.1:42485, datanodeUuid=34ec2e2b-8100-4f9f-8f8d-fa733114e19c, infoPort=39807, infoSecurePort=0, ipcPort=38313, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:12,197 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5fc5711379d2eafa: Processing first storage report for DS-059698ed-f98a-442d-9bb5-5916b90a7180 from datanode 34ec2e2b-8100-4f9f-8f8d-fa733114e19c 2023-09-04 07:58:12,197 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5fc5711379d2eafa: from storage DS-059698ed-f98a-442d-9bb5-5916b90a7180 node DatanodeRegistration(127.0.0.1:42485, datanodeUuid=34ec2e2b-8100-4f9f-8f8d-fa733114e19c, infoPort=39807, infoSecurePort=0, ipcPort=38313, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:12,236 INFO [Listener at localhost/38313] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40547 2023-09-04 07:58:12,246 WARN [Listener at localhost/46575] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:12,310 WARN [Listener at localhost/46575] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:58:12,317 WARN [Listener at localhost/46575] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:12,319 INFO [Listener at localhost/46575] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:12,338 INFO [Listener at localhost/46575] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/java.io.tmpdir/Jetty_localhost_34751_datanode____dtl5a1/webapp 2023-09-04 07:58:12,413 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x38fc9c49d54f1c8a: Processing first storage report for DS-c32e32e9-ad20-49bd-a4a4-9a28064457f5 from datanode 4b98986e-8592-46d3-a7e9-e222d518c2eb 2023-09-04 07:58:12,413 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x38fc9c49d54f1c8a: from storage DS-c32e32e9-ad20-49bd-a4a4-9a28064457f5 node DatanodeRegistration(127.0.0.1:33167, datanodeUuid=4b98986e-8592-46d3-a7e9-e222d518c2eb, infoPort=42997, infoSecurePort=0, ipcPort=46575, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:12,414 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x38fc9c49d54f1c8a: Processing first storage report for DS-932a2fac-0fec-4442-9dcb-24e037f8bfe6 from datanode 4b98986e-8592-46d3-a7e9-e222d518c2eb 2023-09-04 07:58:12,414 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x38fc9c49d54f1c8a: from storage DS-932a2fac-0fec-4442-9dcb-24e037f8bfe6 node DatanodeRegistration(127.0.0.1:33167, datanodeUuid=4b98986e-8592-46d3-a7e9-e222d518c2eb, infoPort=42997, infoSecurePort=0, ipcPort=46575, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:12,451 INFO [Listener at localhost/46575] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34751 2023-09-04 07:58:12,460 WARN [Listener at localhost/34303] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:12,562 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x441f0c4bf056c76e: Processing first storage report for DS-04efb90f-025d-4d29-a5a1-8dbc9ad8e744 from datanode fce1fe4c-d4a9-446b-8911-850892947bd0 2023-09-04 07:58:12,562 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x441f0c4bf056c76e: from storage DS-04efb90f-025d-4d29-a5a1-8dbc9ad8e744 node DatanodeRegistration(127.0.0.1:35835, datanodeUuid=fce1fe4c-d4a9-446b-8911-850892947bd0, infoPort=34135, infoSecurePort=0, ipcPort=34303, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:12,562 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x441f0c4bf056c76e: Processing first storage report for DS-7c923e55-45eb-42ab-900d-105cb968ea18 from datanode fce1fe4c-d4a9-446b-8911-850892947bd0 2023-09-04 07:58:12,562 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x441f0c4bf056c76e: from storage DS-7c923e55-45eb-42ab-900d-105cb968ea18 node DatanodeRegistration(127.0.0.1:35835, datanodeUuid=fce1fe4c-d4a9-446b-8911-850892947bd0, infoPort=34135, infoSecurePort=0, ipcPort=34303, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:12,575 DEBUG [Listener at localhost/34303] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-04 07:58:12,578 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51662, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-04 07:58:12,587 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33813] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-04 07:58:12,587 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33813] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-04 07:58:12,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33813] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 07:58:12,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33813] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath 2023-09-04 07:58:12,595 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 07:58:12,596 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 07:58:12,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33813] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRollOnDatanodeDeath" procId is: 9 2023-09-04 07:58:12,599 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:12,601 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10 empty. 2023-09-04 07:58:12,603 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:12,603 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRollOnDatanodeDeath regions 2023-09-04 07:58:12,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33813] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 07:58:12,626 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp/data/default/TestLogRolling-testLogRollOnDatanodeDeath/.tabledesc/.tableinfo.0000000001 2023-09-04 07:58:12,628 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(7675): creating {ENCODED => 65dfc14a91aabbd73d629601babc2d10, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRollOnDatanodeDeath', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/.tmp 2023-09-04 07:58:12,643 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:12,643 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1604): Closing 65dfc14a91aabbd73d629601babc2d10, disabling compactions & flushes 2023-09-04 07:58:12,643 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:12,643 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:12,643 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. after waiting 0 ms 2023-09-04 07:58:12,643 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:12,643 INFO [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:12,643 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRollOnDatanodeDeath-pool-0] regionserver.HRegion(1558): Region close journal for 65dfc14a91aabbd73d629601babc2d10: 2023-09-04 07:58:12,647 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 07:58:12,649 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693814292649"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814292649"}]},"ts":"1693814292649"} 2023-09-04 07:58:12,652 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 07:58:12,653 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 07:58:12,653 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814292653"}]},"ts":"1693814292653"} 2023-09-04 07:58:12,655 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLING in hbase:meta 2023-09-04 07:58:12,661 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=65dfc14a91aabbd73d629601babc2d10, ASSIGN}] 2023-09-04 07:58:12,663 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=65dfc14a91aabbd73d629601babc2d10, ASSIGN 2023-09-04 07:58:12,664 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=65dfc14a91aabbd73d629601babc2d10, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42995,1693814289190; forceNewPlan=false, retain=false 2023-09-04 07:58:12,816 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=65dfc14a91aabbd73d629601babc2d10, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:12,817 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693814292816"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814292816"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814292816"}]},"ts":"1693814292816"} 2023-09-04 07:58:12,820 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 65dfc14a91aabbd73d629601babc2d10, server=jenkins-hbase4.apache.org,42995,1693814289190}] 2023-09-04 07:58:12,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:12,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 65dfc14a91aabbd73d629601babc2d10, NAME => 'TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:58:12,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRollOnDatanodeDeath 65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:12,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:12,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:12,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:12,984 INFO [StoreOpener-65dfc14a91aabbd73d629601babc2d10-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:12,986 DEBUG [StoreOpener-65dfc14a91aabbd73d629601babc2d10-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10/info 2023-09-04 07:58:12,986 DEBUG [StoreOpener-65dfc14a91aabbd73d629601babc2d10-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10/info 2023-09-04 07:58:12,987 INFO [StoreOpener-65dfc14a91aabbd73d629601babc2d10-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 65dfc14a91aabbd73d629601babc2d10 columnFamilyName info 2023-09-04 07:58:12,988 INFO [StoreOpener-65dfc14a91aabbd73d629601babc2d10-1] regionserver.HStore(310): Store=65dfc14a91aabbd73d629601babc2d10/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:12,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:12,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:12,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:13,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:58:13,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 65dfc14a91aabbd73d629601babc2d10; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=792268, jitterRate=0.007421925663948059}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:58:13,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 65dfc14a91aabbd73d629601babc2d10: 2023-09-04 07:58:13,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10., pid=11, masterSystemTime=1693814292974 2023-09-04 07:58:13,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:13,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:13,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=65dfc14a91aabbd73d629601babc2d10, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:13,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1693814293006"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814293006"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814293006"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814293006"}]},"ts":"1693814293006"} 2023-09-04 07:58:13,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-04 07:58:13,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 65dfc14a91aabbd73d629601babc2d10, server=jenkins-hbase4.apache.org,42995,1693814289190 in 189 msec 2023-09-04 07:58:13,015 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-04 07:58:13,016 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRollOnDatanodeDeath, region=65dfc14a91aabbd73d629601babc2d10, ASSIGN in 352 msec 2023-09-04 07:58:13,017 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 07:58:13,017 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRollOnDatanodeDeath","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814293017"}]},"ts":"1693814293017"} 2023-09-04 07:58:13,019 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRollOnDatanodeDeath, state=ENABLED in hbase:meta 2023-09-04 07:58:13,022 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 07:58:13,024 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRollOnDatanodeDeath in 431 msec 2023-09-04 07:58:16,389 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 07:58:16,452 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-04 07:58:16,453 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-04 07:58:16,454 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRollOnDatanodeDeath' 2023-09-04 07:58:18,147 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 07:58:18,147 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-09-04 07:58:22,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33813] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 07:58:22,615 INFO [Listener at localhost/34303] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRollOnDatanodeDeath, procId: 9 completed 2023-09-04 07:58:22,615 DEBUG [Listener at localhost/34303] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestLogRolling-testLogRollOnDatanodeDeath get assigned. Timeout = 60000ms 2023-09-04 07:58:22,616 INFO [Listener at localhost/34303] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-09-04 07:58:22,621 INFO [Listener at localhost/34303] hbase.HBaseTestingUtility(3484): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned to meta. Checking AM states. 2023-09-04 07:58:22,621 INFO [Listener at localhost/34303] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-09-04 07:58:22,621 INFO [Listener at localhost/34303] hbase.HBaseTestingUtility(3504): All regions for table TestLogRolling-testLogRollOnDatanodeDeath assigned. 2023-09-04 07:58:22,631 DEBUG [Listener at localhost/34303] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRollOnDatanodeDeath 2023-09-04 07:58:22,632 DEBUG [Listener at localhost/34303] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:22,633 WARN [Listener at localhost/34303] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 07:58:22,636 INFO [Listener at localhost/34303] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:58:22,641 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1299937740_17 at /127.0.0.1:46688 [Receiving block BP-307725532-172.31.14.131-1693814284767:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:39997:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:46688 dst: /127.0.0.1:39997 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,641 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1299937740_17 at /127.0.0.1:46682 [Receiving block BP-307725532-172.31.14.131-1693814284767:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:39997:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:46682 dst: /127.0.0.1:39997 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,641 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-196406040_17 at /127.0.0.1:46644 [Receiving block BP-307725532-172.31.14.131-1693814284767:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:39997:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:46644 dst: /127.0.0.1:39997 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,738 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1299937740_17 at /127.0.0.1:39226 [Receiving block BP-307725532-172.31.14.131-1693814284767:blk_1073741833_1009]] datanode.DataXceiver(323): 127.0.0.1:34053:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:39226 dst: /127.0.0.1:34053 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,739 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1299937740_17 at /127.0.0.1:39210 [Receiving block BP-307725532-172.31.14.131-1693814284767:blk_1073741832_1008]] datanode.DataXceiver(323): 127.0.0.1:34053:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:39210 dst: /127.0.0.1:34053 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,740 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 07:58:22,738 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-196406040_17 at /127.0.0.1:39172 [Receiving block BP-307725532-172.31.14.131-1693814284767:blk_1073741829_1005]] datanode.DataXceiver(323): 127.0.0.1:34053:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:39172 dst: /127.0.0.1:34053 java.io.InterruptedIOException: Interrupted while waiting for IO on channel java.nio.channels.SocketChannel[closed]. 60000 millis timeout left. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:342) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,741 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-307725532-172.31.14.131-1693814284767 (Datanode Uuid 7d45ad25-d115-483f-b028-6ae759d392aa) service to localhost/127.0.0.1:39519 2023-09-04 07:58:22,743 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data1/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:22,743 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data2/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:22,750 WARN [Listener at localhost/34303] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:58:22,752 WARN [Listener at localhost/34303] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:22,753 INFO [Listener at localhost/34303] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:22,758 INFO [Listener at localhost/34303] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/java.io.tmpdir/Jetty_localhost_44721_datanode____.9qt5nn/webapp 2023-09-04 07:58:22,850 INFO [Listener at localhost/34303] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44721 2023-09-04 07:58:22,858 WARN [Listener at localhost/45229] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:22,862 WARN [AsyncFSWAL-0-hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb-prefix:jenkins-hbase4.apache.org,42995,1693814289190] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,864 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C42995%2C1693814289190:(num 1693814290584) roll requested 2023-09-04 07:58:22,881 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35835,DS-04efb90f-025d-4d29-a5a1-8dbc9ad8e744,DISK] 2023-09-04 07:58:22,881 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42485,DS-0d033ba1-87bf-4942-8573-cd8f66247aff,DISK] 2023-09-04 07:58:22,886 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 with entries=6, filesize=3.19 KB; new WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814302864 2023-09-04 07:58:22,887 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35835,DS-04efb90f-025d-4d29-a5a1-8dbc9ad8e744,DISK], DatanodeInfoWithStorage[127.0.0.1:42485,DS-0d033ba1-87bf-4942-8573-cd8f66247aff,DISK]] 2023-09-04 07:58:22,887 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 is not closed yet, will try archiving it next time 2023-09-04 07:58:22,888 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-04 07:58:22,888 INFO [Listener at localhost/45229] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-04 07:58:22,890 DEBUG [Listener at localhost/45229] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x42bfd752 to 127.0.0.1:58270 2023-09-04 07:58:22,890 DEBUG [Listener at localhost/45229] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:58:22,890 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,891 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,891 DEBUG [Listener at localhost/45229] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-04 07:58:22,891 DEBUG [Listener at localhost/45229] util.JVMClusterUtil(257): Found active master hash=335748510, stopped=false 2023-09-04 07:58:22,891 INFO [Listener at localhost/45229] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:22,893 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 2023-09-04 07:58:22,894 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 07:58:22,894 INFO [Listener at localhost/45229] procedure2.ProcedureExecutor(628): Stopping 2023-09-04 07:58:22,894 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 07:58:22,894 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:22,894 DEBUG [Listener at localhost/45229] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x023effef to 127.0.0.1:58270 2023-09-04 07:58:22,895 DEBUG [Listener at localhost/45229] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:58:22,896 INFO [Listener at localhost/45229] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42995,1693814289190' ***** 2023-09-04 07:58:22,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:58:22,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:58:22,896 INFO [Listener at localhost/45229] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-04 07:58:22,897 INFO [RS:0;jenkins-hbase4:42995] regionserver.HeapMemoryManager(220): Stopping 2023-09-04 07:58:22,897 INFO [RS:0;jenkins-hbase4:42995] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-04 07:58:22,897 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-04 07:58:22,897 INFO [RS:0;jenkins-hbase4:42995] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-04 07:58:22,898 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(3305): Received CLOSE for 65dfc14a91aabbd73d629601babc2d10 2023-09-04 07:58:22,899 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(3305): Received CLOSE for 6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:22,899 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:22,899 DEBUG [RS:0;jenkins-hbase4:42995] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67e7c1db to 127.0.0.1:58270 2023-09-04 07:58:22,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 65dfc14a91aabbd73d629601babc2d10, disabling compactions & flushes 2023-09-04 07:58:22,899 DEBUG [RS:0;jenkins-hbase4:42995] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:58:22,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:22,900 INFO [RS:0;jenkins-hbase4:42995] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-04 07:58:22,900 INFO [RS:0;jenkins-hbase4:42995] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-04 07:58:22,900 INFO [RS:0;jenkins-hbase4:42995] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-04 07:58:22,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:22,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. after waiting 0 ms 2023-09-04 07:58:22,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:22,900 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-04 07:58:22,900 WARN [IPC Server handler 3 on default port 39519] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 has not been closed. Lease recovery is in progress. RecoveryId = 1015 for block blk_1073741832_1008 2023-09-04 07:58:22,900 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-09-04 07:58:22,900 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1478): Online Regions={65dfc14a91aabbd73d629601babc2d10=TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10., 1588230740=hbase:meta,,1.1588230740, 6b6bea4f63cc581dcd842e858450946a=hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a.} 2023-09-04 07:58:22,901 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 07:58:22,901 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 07:58:22,901 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 07:58:22,901 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 07:58:22,901 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 07:58:22,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 65dfc14a91aabbd73d629601babc2d10 1/1 column families, dataSize=2.05 KB heapSize=2.45 KB 2023-09-04 07:58:22,901 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.92 KB heapSize=5.45 KB 2023-09-04 07:58:22,903 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1504): Waiting on 1588230740, 65dfc14a91aabbd73d629601babc2d10, 6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:22,903 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 after 9ms 2023-09-04 07:58:22,911 WARN [AsyncFSWAL-0-hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb-prefix:jenkins-hbase4.apache.org,42995,1693814289190.meta] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,912 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta:.meta(num 1693814290816) roll requested 2023-09-04 07:58:22,936 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33167,DS-c32e32e9-ad20-49bd-a4a4-9a28064457f5,DISK] 2023-09-04 07:58:22,936 WARN [regionserver/jenkins-hbase4:0.logRoller] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(571): create fan-out dfs output /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814302912.meta failed, retry = 0 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: /127.0.0.1:34053 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:22,971 WARN [Thread-334] hdfs.DataStreamer(1658): Abandoning BP-307725532-172.31.14.131-1693814284767:blk_1073741840_1017 2023-09-04 07:58:22,975 WARN [Thread-334] hdfs.DataStreamer(1663): Excluding datanode DatanodeInfoWithStorage[127.0.0.1:34053,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK] 2023-09-04 07:58:22,986 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x34e21cdfbe21b4a2: Processing first storage report for DS-ac57ca98-79e4-40e5-a55a-812fa81554a4 from datanode 7d45ad25-d115-483f-b028-6ae759d392aa 2023-09-04 07:58:22,988 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x34e21cdfbe21b4a2: from storage DS-ac57ca98-79e4-40e5-a55a-812fa81554a4 node DatanodeRegistration(127.0.0.1:33223, datanodeUuid=7d45ad25-d115-483f-b028-6ae759d392aa, infoPort=34223, infoSecurePort=0, ipcPort=45229, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 6, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-04 07:58:22,988 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x34e21cdfbe21b4a2: Processing first storage report for DS-bd10e2cd-96cf-4943-b6bf-4f6634ef4805 from datanode 7d45ad25-d115-483f-b028-6ae759d392aa 2023-09-04 07:58:22,989 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x34e21cdfbe21b4a2: from storage DS-bd10e2cd-96cf-4943-b6bf-4f6634ef4805 node DatanodeRegistration(127.0.0.1:33223, datanodeUuid=7d45ad25-d115-483f-b028-6ae759d392aa, infoPort=34223, infoSecurePort=0, ipcPort=45229, storageInfo=lv=-57;cid=testClusterID;nsid=1709424695;c=1693814284767), blocks: 5, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:22,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.05 KB at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10/.tmp/info/6a879e5c371844a0b8b028103438c91c 2023-09-04 07:58:23,040 ERROR [DataXceiver for client DFSClient_NONMAPREDUCE_-1299937740_17 at /127.0.0.1:56444 [Receiving block BP-307725532-172.31.14.131-1693814284767:blk_1073741839_1016]] datanode.DataXceiver(323): 127.0.0.1:33167:DataXceiver error processing WRITE_BLOCK operation src: /127.0.0.1:56444 dst: /127.0.0.1:33167 java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:23,044 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42485,DS-0d033ba1-87bf-4942-8573-cd8f66247aff,DISK] 2023-09-04 07:58:23,046 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33167,DS-c32e32e9-ad20-49bd-a4a4-9a28064457f5,DISK] 2023-09-04 07:58:23,059 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta with entries=12, filesize=3.87 KB; new WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814302912.meta 2023-09-04 07:58:23,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10/.tmp/info/6a879e5c371844a0b8b028103438c91c as hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10/info/6a879e5c371844a0b8b028103438c91c 2023-09-04 07:58:23,061 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42485,DS-0d033ba1-87bf-4942-8573-cd8f66247aff,DISK], DatanodeInfoWithStorage[127.0.0.1:33167,DS-c32e32e9-ad20-49bd-a4a4-9a28064457f5,DISK]] 2023-09-04 07:58:23,061 DEBUG [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractFSWAL(716): hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta is not closed yet, will try archiving it next time 2023-09-04 07:58:23,062 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:23,064 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:23,064 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta 2023-09-04 07:58:23,065 WARN [IPC Server handler 4 on default port 39519] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta has not been closed. Lease recovery is in progress. RecoveryId = 1020 for block blk_1073741833_1009 2023-09-04 07:58:23,066 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta after 2ms 2023-09-04 07:58:23,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10/info/6a879e5c371844a0b8b028103438c91c, entries=2, sequenceid=6, filesize=6.8 K 2023-09-04 07:58:23,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.05 KB/2102, heapSize ~2.44 KB/2496, currentSize=0 B/0 for 65dfc14a91aabbd73d629601babc2d10 in 183ms, sequenceid=6, compaction requested=false 2023-09-04 07:58:23,092 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.69 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/.tmp/info/c4b45c7b3e3148f0830680768ec2f243 2023-09-04 07:58:23,103 DEBUG [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1504): Waiting on 1588230740, 65dfc14a91aabbd73d629601babc2d10, 6b6bea4f63cc581dcd842e858450946a 2023-09-04 07:58:23,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/default/TestLogRolling-testLogRollOnDatanodeDeath/65dfc14a91aabbd73d629601babc2d10/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 07:58:23,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:23,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 65dfc14a91aabbd73d629601babc2d10: 2023-09-04 07:58:23,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRollOnDatanodeDeath,,1693814292587.65dfc14a91aabbd73d629601babc2d10. 2023-09-04 07:58:23,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6b6bea4f63cc581dcd842e858450946a, disabling compactions & flushes 2023-09-04 07:58:23,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:23,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:23,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. after waiting 0 ms 2023-09-04 07:58:23,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:23,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6b6bea4f63cc581dcd842e858450946a 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-04 07:58:23,136 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/.tmp/table/3de099a34e914f3daf98e43eea599a8a 2023-09-04 07:58:23,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a/.tmp/info/7440e1e80a0843518f44feecf593d8b7 2023-09-04 07:58:23,156 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/.tmp/info/c4b45c7b3e3148f0830680768ec2f243 as hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/info/c4b45c7b3e3148f0830680768ec2f243 2023-09-04 07:58:23,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a/.tmp/info/7440e1e80a0843518f44feecf593d8b7 as hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a/info/7440e1e80a0843518f44feecf593d8b7 2023-09-04 07:58:23,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a/info/7440e1e80a0843518f44feecf593d8b7, entries=2, sequenceid=6, filesize=4.8 K 2023-09-04 07:58:23,169 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/info/c4b45c7b3e3148f0830680768ec2f243, entries=20, sequenceid=14, filesize=7.4 K 2023-09-04 07:58:23,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6b6bea4f63cc581dcd842e858450946a in 63ms, sequenceid=6, compaction requested=false 2023-09-04 07:58:23,171 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/.tmp/table/3de099a34e914f3daf98e43eea599a8a as hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/table/3de099a34e914f3daf98e43eea599a8a 2023-09-04 07:58:23,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/namespace/6b6bea4f63cc581dcd842e858450946a/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 07:58:23,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:23,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6b6bea4f63cc581dcd842e858450946a: 2023-09-04 07:58:23,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693814291060.6b6bea4f63cc581dcd842e858450946a. 2023-09-04 07:58:23,185 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/table/3de099a34e914f3daf98e43eea599a8a, entries=4, sequenceid=14, filesize=4.8 K 2023-09-04 07:58:23,186 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.92 KB/2994, heapSize ~5.16 KB/5288, currentSize=0 B/0 for 1588230740 in 285ms, sequenceid=14, compaction requested=false 2023-09-04 07:58:23,194 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-09-04 07:58:23,195 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 07:58:23,196 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 07:58:23,196 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 07:58:23,196 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-04 07:58:23,303 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42995,1693814289190; all regions closed. 2023-09-04 07:58:23,470 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-09-04 07:58:23,470 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-09-04 07:58:24,433 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-04 07:58:26,904 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 after 4010ms 2023-09-04 07:58:26,906 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/oldWALs/jenkins-hbase4.apache.org%2C42995%2C1693814289190.1693814290584 2023-09-04 07:58:27,067 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta after 4003ms 2023-09-04 07:58:27,068 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/WALs/jenkins-hbase4.apache.org,42995,1693814289190/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/oldWALs/jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta.1693814290816.meta 2023-09-04 07:58:27,073 DEBUG [RS:0;jenkins-hbase4:42995] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/oldWALs 2023-09-04 07:58:27,073 INFO [RS:0;jenkins-hbase4:42995] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42995%2C1693814289190.meta:.meta(num 1693814302912) 2023-09-04 07:58:27,082 DEBUG [RS:0;jenkins-hbase4:42995] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/oldWALs 2023-09-04 07:58:27,082 INFO [RS:0;jenkins-hbase4:42995] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42995%2C1693814289190:(num 1693814302864) 2023-09-04 07:58:27,082 DEBUG [RS:0;jenkins-hbase4:42995] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:58:27,082 INFO [RS:0;jenkins-hbase4:42995] regionserver.LeaseManager(133): Closed leases 2023-09-04 07:58:27,082 INFO [RS:0;jenkins-hbase4:42995] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-09-04 07:58:27,082 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 07:58:27,083 INFO [RS:0;jenkins-hbase4:42995] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42995 2023-09-04 07:58:27,090 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42995,1693814289190 2023-09-04 07:58:27,090 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 07:58:27,090 ERROR [Listener at localhost/35195-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73d89a3f rejected from java.util.concurrent.ThreadPoolExecutor@77e7109c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1374) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:603) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-09-04 07:58:27,091 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 07:58:27,091 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42995,1693814289190] 2023-09-04 07:58:27,092 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42995,1693814289190; numProcessing=1 2023-09-04 07:58:27,094 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42995,1693814289190 already deleted, retry=false 2023-09-04 07:58:27,094 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42995,1693814289190 expired; onlineServers=0 2023-09-04 07:58:27,094 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33813,1693814287978' ***** 2023-09-04 07:58:27,094 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-04 07:58:27,095 DEBUG [M:0;jenkins-hbase4:33813] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@553e17a5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 07:58:27,095 INFO [M:0;jenkins-hbase4:33813] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33813,1693814287978 2023-09-04 07:58:27,095 INFO [M:0;jenkins-hbase4:33813] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33813,1693814287978; all regions closed. 2023-09-04 07:58:27,095 DEBUG [M:0;jenkins-hbase4:33813] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:58:27,095 DEBUG [M:0;jenkins-hbase4:33813] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-04 07:58:27,095 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-04 07:58:27,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814290281] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814290281,5,FailOnTimeoutGroup] 2023-09-04 07:58:27,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814290276] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814290276,5,FailOnTimeoutGroup] 2023-09-04 07:58:27,095 DEBUG [M:0;jenkins-hbase4:33813] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-04 07:58:27,097 INFO [M:0;jenkins-hbase4:33813] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-04 07:58:27,097 INFO [M:0;jenkins-hbase4:33813] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-04 07:58:27,097 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-04 07:58:27,097 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:27,097 INFO [M:0;jenkins-hbase4:33813] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-04 07:58:27,097 DEBUG [M:0;jenkins-hbase4:33813] master.HMaster(1512): Stopping service threads 2023-09-04 07:58:27,097 INFO [M:0;jenkins-hbase4:33813] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-04 07:58:27,098 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 07:58:27,098 ERROR [M:0;jenkins-hbase4:33813] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] 2023-09-04 07:58:27,098 INFO [M:0;jenkins-hbase4:33813] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-04 07:58:27,098 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-04 07:58:27,099 DEBUG [M:0;jenkins-hbase4:33813] zookeeper.ZKUtil(398): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-04 07:58:27,099 WARN [M:0;jenkins-hbase4:33813] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-04 07:58:27,099 INFO [M:0;jenkins-hbase4:33813] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-04 07:58:27,099 INFO [M:0;jenkins-hbase4:33813] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-04 07:58:27,100 DEBUG [M:0;jenkins-hbase4:33813] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 07:58:27,100 INFO [M:0;jenkins-hbase4:33813] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:27,100 DEBUG [M:0;jenkins-hbase4:33813] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:27,100 DEBUG [M:0;jenkins-hbase4:33813] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 07:58:27,100 DEBUG [M:0;jenkins-hbase4:33813] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:27,100 INFO [M:0;jenkins-hbase4:33813] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.08 KB heapSize=45.73 KB 2023-09-04 07:58:27,101 WARN [AsyncFSWAL-0-hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData-prefix:jenkins-hbase4.apache.org,33813,1693814287978] wal.AsyncFSWAL(305): sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:159) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:419) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:631) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:27,101 DEBUG [master:store-WAL-Roller] wal.AbstractWALRoller(197): WAL AsyncFSWAL jenkins-hbase4.apache.org%2C33813%2C1693814287978:(num 1693814289629) roll requested 2023-09-04 07:58:27,120 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42485,DS-0d033ba1-87bf-4942-8573-cd8f66247aff,DISK] 2023-09-04 07:58:27,121 DEBUG [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33223,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK] 2023-09-04 07:58:27,126 INFO [master:store-WAL-Roller] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 with entries=89, filesize=43.93 KB; new WAL /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814307101 2023-09-04 07:58:27,126 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42485,DS-0d033ba1-87bf-4942-8573-cd8f66247aff,DISK], DatanodeInfoWithStorage[127.0.0.1:33223,DS-ac57ca98-79e4-40e5-a55a-812fa81554a4,DISK]] 2023-09-04 07:58:27,126 DEBUG [master:store-WAL-Roller] wal.AbstractFSWAL(716): hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 is not closed yet, will try archiving it next time 2023-09-04 07:58:27,126 WARN [Close-WAL-Writer-0] wal.AbstractProtobufLogWriter(237): Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:436) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:525) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:252) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:208) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:243) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:234) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:168) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:27,127 WARN [Close-WAL-Writer-0] wal.AsyncProtobufLogWriter(171): normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:533) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:581) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:169) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$5(AsyncFSWAL.java:746) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-09-04 07:58:27,127 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(86): Recover lease on dfs file /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 2023-09-04 07:58:27,128 WARN [IPC Server handler 1 on default port 39519] namenode.FSNamesystem(3291): DIR* NameSystem.internalReleaseLease: File /user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 has not been closed. Lease recovery is in progress. RecoveryId = 1025 for block blk_1073741829_1005 2023-09-04 07:58:27,128 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Failed to recover lease, attempt=0 on file=/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 after 1ms 2023-09-04 07:58:27,142 INFO [M:0;jenkins-hbase4:33813] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.08 KB at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/58c1071ca951401e9fae344dbc0e934a 2023-09-04 07:58:27,152 DEBUG [M:0;jenkins-hbase4:33813] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/58c1071ca951401e9fae344dbc0e934a as hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/58c1071ca951401e9fae344dbc0e934a 2023-09-04 07:58:27,159 INFO [M:0;jenkins-hbase4:33813] regionserver.HStore(1080): Added hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/58c1071ca951401e9fae344dbc0e934a, entries=11, sequenceid=91, filesize=7.0 K 2023-09-04 07:58:27,160 INFO [M:0;jenkins-hbase4:33813] regionserver.HRegion(2948): Finished flush of dataSize ~38.08 KB/38997, heapSize ~45.72 KB/46816, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 60ms, sequenceid=91, compaction requested=false 2023-09-04 07:58:27,162 INFO [M:0;jenkins-hbase4:33813] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:27,162 DEBUG [M:0;jenkins-hbase4:33813] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 07:58:27,192 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 07:58:27,192 INFO [RS:0;jenkins-hbase4:42995] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42995,1693814289190; zookeeper connection closed. 2023-09-04 07:58:27,192 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): regionserver:42995-0x10075f222ab0001, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 07:58:27,193 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@110b820b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@110b820b 2023-09-04 07:58:27,193 INFO [Listener at localhost/45229] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-04 07:58:28,641 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 07:58:28,681 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 07:58:31,129 INFO [Close-WAL-Writer-0] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=1 on file=/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 after 4002ms 2023-09-04 07:58:31,130 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/WALs/jenkins-hbase4.apache.org,33813,1693814287978/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/oldWALs/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 2023-09-04 07:58:31,136 INFO [WAL-Archive-0] region.MasterRegionUtils(50): Moved hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/MasterData/oldWALs/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629 to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/oldWALs/jenkins-hbase4.apache.org%2C33813%2C1693814287978.1693814289629$masterlocalwal$ 2023-09-04 07:58:31,136 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 07:58:31,136 INFO [M:0;jenkins-hbase4:33813] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-04 07:58:31,137 INFO [M:0;jenkins-hbase4:33813] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33813 2023-09-04 07:58:31,140 DEBUG [M:0;jenkins-hbase4:33813] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33813,1693814287978 already deleted, retry=false 2023-09-04 07:58:31,243 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 07:58:31,243 INFO [M:0;jenkins-hbase4:33813] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33813,1693814287978; zookeeper connection closed. 2023-09-04 07:58:31,243 DEBUG [Listener at localhost/35195-EventThread] zookeeper.ZKWatcher(600): master:33813-0x10075f222ab0000, quorum=127.0.0.1:58270, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 07:58:31,245 WARN [Listener at localhost/45229] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 07:58:31,248 INFO [Listener at localhost/45229] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:58:31,353 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 07:58:31,353 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-307725532-172.31.14.131-1693814284767 (Datanode Uuid 7d45ad25-d115-483f-b028-6ae759d392aa) service to localhost/127.0.0.1:39519 2023-09-04 07:58:31,354 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data1/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,354 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data2/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,356 WARN [Listener at localhost/45229] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 07:58:31,362 INFO [Listener at localhost/45229] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:58:31,465 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 07:58:31,466 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-307725532-172.31.14.131-1693814284767 (Datanode Uuid fce1fe4c-d4a9-446b-8911-850892947bd0) service to localhost/127.0.0.1:39519 2023-09-04 07:58:31,466 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data9/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,467 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data10/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,468 WARN [Listener at localhost/45229] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 07:58:31,471 INFO [Listener at localhost/45229] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:58:31,575 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 07:58:31,575 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-307725532-172.31.14.131-1693814284767 (Datanode Uuid 4b98986e-8592-46d3-a7e9-e222d518c2eb) service to localhost/127.0.0.1:39519 2023-09-04 07:58:31,576 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data7/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,576 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data8/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,577 WARN [Listener at localhost/45229] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 07:58:31,580 INFO [Listener at localhost/45229] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:58:31,684 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 07:58:31,684 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-307725532-172.31.14.131-1693814284767 (Datanode Uuid 34ec2e2b-8100-4f9f-8f8d-fa733114e19c) service to localhost/127.0.0.1:39519 2023-09-04 07:58:31,685 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data5/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,686 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data6/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,687 WARN [Listener at localhost/45229] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 07:58:31,690 INFO [Listener at localhost/45229] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:58:31,791 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 07:58:31,792 WARN [BP-307725532-172.31.14.131-1693814284767 heartbeating to localhost/127.0.0.1:39519] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-307725532-172.31.14.131-1693814284767 (Datanode Uuid 71fc19a9-265d-4ab6-b211-2890fd2d7ee2) service to localhost/127.0.0.1:39519 2023-09-04 07:58:31,792 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data3/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,792 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/cluster_c3941fcb-2470-0ab9-9cbc-b7bf5888dacb/dfs/data/data4/current/BP-307725532-172.31.14.131-1693814284767] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:58:31,823 INFO [Listener at localhost/45229] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:58:31,939 INFO [Listener at localhost/45229] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-04 07:58:31,982 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-04 07:58:31,994 INFO [Listener at localhost/45229] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnDatanodeDeath Thread=78 (was 10) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.0@localhost:39519 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: region-location-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener at localhost/45229 java.lang.Thread.dumpThreads(Native Method) java.lang.Thread.getAllStackTraces(Thread.java:1615) org.apache.hadoop.hbase.ResourceCheckerJUnitListener$ThreadResourceAnalyzer.getVal(ResourceCheckerJUnitListener.java:49) org.apache.hadoop.hbase.ResourceChecker.fill(ResourceChecker.java:110) org.apache.hadoop.hbase.ResourceChecker.fillEndings(ResourceChecker.java:104) org.apache.hadoop.hbase.ResourceChecker.end(ResourceChecker.java:206) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.end(ResourceCheckerJUnitListener.java:165) org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:185) org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87) org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SessionTracker java.lang.Thread.sleep(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Potentially hanging thread: ForkJoinPool-2-worker-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:39519 from jenkins.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-6-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'DataNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-1 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: regionserver/jenkins-hbase4:0.procedureResultReporter sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:77) Potentially hanging thread: nioEventLoopGroup-10-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-10-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:39519 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: nioEventLoopGroup-5-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-11-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-5-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-10-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ForkJoinPool-2-worker-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Potentially hanging thread: nioEventLoopGroup-11-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: SnapshotHandlerChoreCleaner sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-7-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins@localhost:39519 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-13-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-4-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-9-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-3 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-12-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-6-2 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@1e8dc4ec java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: nioEventLoopGroup-8-1 java.lang.Thread.sleep(Native Method) io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:790) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:525) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (524694476) connection to localhost/127.0.0.1:39519 from jenkins java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) - Thread LEAK? -, OpenFileDescriptor=467 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=135 (was 214), ProcessCount=167 (was 167), AvailableMemoryMB=8849 (was 9722) 2023-09-04 07:58:32,003 INFO [Listener at localhost/45229] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=78, OpenFileDescriptor=467, MaxFileDescriptor=60000, SystemLoadAverage=135, ProcessCount=167, AvailableMemoryMB=8848 2023-09-04 07:58:32,004 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-04 07:58:32,004 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/hadoop.log.dir so I do NOT create it in target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6 2023-09-04 07:58:32,004 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ff007c15-793a-e1bc-0d3a-39c53e5c0db6/hadoop.tmp.dir so I do NOT create it in target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6 2023-09-04 07:58:32,004 INFO [Listener at localhost/45229] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/cluster_d0389aae-903d-8a9a-817d-7948e87b3c60, deleteOnExit=true 2023-09-04 07:58:32,004 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-04 07:58:32,004 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/test.cache.data in system properties and HBase conf 2023-09-04 07:58:32,004 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/hadoop.tmp.dir in system properties and HBase conf 2023-09-04 07:58:32,004 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/hadoop.log.dir in system properties and HBase conf 2023-09-04 07:58:32,005 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-04 07:58:32,005 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-04 07:58:32,005 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-04 07:58:32,005 DEBUG [Listener at localhost/45229] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-04 07:58:32,005 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-04 07:58:32,005 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/nfs.dump.dir in system properties and HBase conf 2023-09-04 07:58:32,006 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/java.io.tmpdir in system properties and HBase conf 2023-09-04 07:58:32,007 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 07:58:32,007 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-04 07:58:32,007 INFO [Listener at localhost/45229] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-04 07:58:32,013 WARN [Listener at localhost/45229] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 07:58:32,013 WARN [Listener at localhost/45229] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 07:58:32,069 WARN [Listener at localhost/45229] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:32,071 INFO [Listener at localhost/45229] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:32,076 INFO [Listener at localhost/45229] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/java.io.tmpdir/Jetty_localhost_45935_hdfs____.gahz4s/webapp 2023-09-04 07:58:32,172 INFO [Listener at localhost/45229] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45935 2023-09-04 07:58:32,178 WARN [Listener at localhost/45229] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 07:58:32,178 WARN [Listener at localhost/45229] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 07:58:32,228 WARN [Listener at localhost/39673] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:32,239 WARN [Listener at localhost/39673] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:58:32,241 WARN [Listener at localhost/39673] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:32,243 INFO [Listener at localhost/39673] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:32,248 INFO [Listener at localhost/39673] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/java.io.tmpdir/Jetty_localhost_38675_datanode____.9zf9u2/webapp 2023-09-04 07:58:32,341 INFO [Listener at localhost/39673] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38675 2023-09-04 07:58:32,348 WARN [Listener at localhost/37539] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:32,360 WARN [Listener at localhost/37539] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:58:32,362 WARN [Listener at localhost/37539] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:58:32,364 INFO [Listener at localhost/37539] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:58:32,368 INFO [Listener at localhost/37539] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/java.io.tmpdir/Jetty_localhost_39059_datanode____.sn8wvf/webapp 2023-09-04 07:58:32,442 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfc7739c053e03a26: Processing first storage report for DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e from datanode bb6f7770-d5af-4835-a80d-4e752f983720 2023-09-04 07:58:32,443 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfc7739c053e03a26: from storage DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e node DatanodeRegistration(127.0.0.1:33895, datanodeUuid=bb6f7770-d5af-4835-a80d-4e752f983720, infoPort=37647, infoSecurePort=0, ipcPort=37539, storageInfo=lv=-57;cid=testClusterID;nsid=956422388;c=1693814312018), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:32,443 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfc7739c053e03a26: Processing first storage report for DS-3d072297-289e-4ac0-8ca9-53133928e0d2 from datanode bb6f7770-d5af-4835-a80d-4e752f983720 2023-09-04 07:58:32,443 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfc7739c053e03a26: from storage DS-3d072297-289e-4ac0-8ca9-53133928e0d2 node DatanodeRegistration(127.0.0.1:33895, datanodeUuid=bb6f7770-d5af-4835-a80d-4e752f983720, infoPort=37647, infoSecurePort=0, ipcPort=37539, storageInfo=lv=-57;cid=testClusterID;nsid=956422388;c=1693814312018), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:32,467 INFO [Listener at localhost/37539] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39059 2023-09-04 07:58:32,474 WARN [Listener at localhost/44221] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:58:32,568 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x253d2368171769e2: Processing first storage report for DS-e2ecdc25-0802-496e-ad7d-59a19951d5db from datanode 9c03b603-7daa-4919-8f7d-d2206dfaadc3 2023-09-04 07:58:32,568 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x253d2368171769e2: from storage DS-e2ecdc25-0802-496e-ad7d-59a19951d5db node DatanodeRegistration(127.0.0.1:34853, datanodeUuid=9c03b603-7daa-4919-8f7d-d2206dfaadc3, infoPort=43661, infoSecurePort=0, ipcPort=44221, storageInfo=lv=-57;cid=testClusterID;nsid=956422388;c=1693814312018), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:32,568 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x253d2368171769e2: Processing first storage report for DS-7b2b793d-2efb-4e70-bf6d-dee7e071f5c2 from datanode 9c03b603-7daa-4919-8f7d-d2206dfaadc3 2023-09-04 07:58:32,568 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x253d2368171769e2: from storage DS-7b2b793d-2efb-4e70-bf6d-dee7e071f5c2 node DatanodeRegistration(127.0.0.1:34853, datanodeUuid=9c03b603-7daa-4919-8f7d-d2206dfaadc3, infoPort=43661, infoSecurePort=0, ipcPort=44221, storageInfo=lv=-57;cid=testClusterID;nsid=956422388;c=1693814312018), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:58:32,582 DEBUG [Listener at localhost/44221] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6 2023-09-04 07:58:32,585 INFO [Listener at localhost/44221] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/cluster_d0389aae-903d-8a9a-817d-7948e87b3c60/zookeeper_0, clientPort=49378, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/cluster_d0389aae-903d-8a9a-817d-7948e87b3c60/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/cluster_d0389aae-903d-8a9a-817d-7948e87b3c60/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-04 07:58:32,590 INFO [Listener at localhost/44221] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=49378 2023-09-04 07:58:32,590 INFO [Listener at localhost/44221] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:32,591 INFO [Listener at localhost/44221] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:32,610 INFO [Listener at localhost/44221] util.FSUtils(471): Created version file at hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc with version=8 2023-09-04 07:58:32,610 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/hbase-staging 2023-09-04 07:58:32,612 INFO [Listener at localhost/44221] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 07:58:32,613 INFO [Listener at localhost/44221] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:32,613 INFO [Listener at localhost/44221] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:32,613 INFO [Listener at localhost/44221] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 07:58:32,613 INFO [Listener at localhost/44221] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:32,613 INFO [Listener at localhost/44221] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 07:58:32,613 INFO [Listener at localhost/44221] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 07:58:32,614 INFO [Listener at localhost/44221] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33811 2023-09-04 07:58:32,615 INFO [Listener at localhost/44221] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:32,617 INFO [Listener at localhost/44221] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:32,618 INFO [Listener at localhost/44221] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33811 connecting to ZooKeeper ensemble=127.0.0.1:49378 2023-09-04 07:58:32,626 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:338110x0, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 07:58:32,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33811-0x10075f2862e0000 connected 2023-09-04 07:58:32,648 DEBUG [Listener at localhost/44221] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 07:58:32,648 DEBUG [Listener at localhost/44221] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:58:32,649 DEBUG [Listener at localhost/44221] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 07:58:32,649 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33811 2023-09-04 07:58:32,650 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33811 2023-09-04 07:58:32,650 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33811 2023-09-04 07:58:32,650 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33811 2023-09-04 07:58:32,650 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33811 2023-09-04 07:58:32,650 INFO [Listener at localhost/44221] master.HMaster(444): hbase.rootdir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc, hbase.cluster.distributed=false 2023-09-04 07:58:32,664 INFO [Listener at localhost/44221] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 07:58:32,664 INFO [Listener at localhost/44221] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:32,664 INFO [Listener at localhost/44221] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:32,664 INFO [Listener at localhost/44221] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 07:58:32,664 INFO [Listener at localhost/44221] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:58:32,665 INFO [Listener at localhost/44221] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 07:58:32,665 INFO [Listener at localhost/44221] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 07:58:32,666 INFO [Listener at localhost/44221] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35915 2023-09-04 07:58:32,667 INFO [Listener at localhost/44221] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-04 07:58:32,668 DEBUG [Listener at localhost/44221] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-04 07:58:32,669 INFO [Listener at localhost/44221] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:32,670 INFO [Listener at localhost/44221] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:32,671 INFO [Listener at localhost/44221] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35915 connecting to ZooKeeper ensemble=127.0.0.1:49378 2023-09-04 07:58:32,680 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:359150x0, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 07:58:32,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35915-0x10075f2862e0001 connected 2023-09-04 07:58:32,681 DEBUG [Listener at localhost/44221] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 07:58:32,682 DEBUG [Listener at localhost/44221] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:58:32,682 DEBUG [Listener at localhost/44221] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 07:58:32,683 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35915 2023-09-04 07:58:32,683 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35915 2023-09-04 07:58:32,685 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35915 2023-09-04 07:58:32,689 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35915 2023-09-04 07:58:32,689 DEBUG [Listener at localhost/44221] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35915 2023-09-04 07:58:32,690 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:58:32,692 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 07:58:32,692 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:58:32,694 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 07:58:32,694 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 07:58:32,694 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:32,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 07:58:32,696 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33811,1693814312612 from backup master directory 2023-09-04 07:58:32,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 07:58:32,697 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:58:32,697 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 07:58:32,697 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 07:58:32,697 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:58:32,719 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/hbase.id with ID: 4e03d1df-b776-47c0-bef2-59ac9d80669c 2023-09-04 07:58:32,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:32,738 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:32,749 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4f27143d to 127.0.0.1:49378 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:58:32,752 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@465343d5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:58:32,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 07:58:32,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-04 07:58:32,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:58:32,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store-tmp 2023-09-04 07:58:32,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:32,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 07:58:32,765 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:32,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:32,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 07:58:32,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:32,765 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:58:32,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 07:58:32,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/WALs/jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:58:32,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33811%2C1693814312612, suffix=, logDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/WALs/jenkins-hbase4.apache.org,33811,1693814312612, archiveDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/oldWALs, maxLogs=10 2023-09-04 07:58:32,785 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK] 2023-09-04 07:58:32,786 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK] 2023-09-04 07:58:32,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/WALs/jenkins-hbase4.apache.org,33811,1693814312612/jenkins-hbase4.apache.org%2C33811%2C1693814312612.1693814312769 2023-09-04 07:58:32,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK], DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK]] 2023-09-04 07:58:32,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:58:32,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:32,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:32,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:32,791 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:32,793 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-04 07:58:32,794 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-04 07:58:32,795 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:32,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:32,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:32,800 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:58:32,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:58:32,803 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=740258, jitterRate=-0.05871422588825226}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:58:32,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 07:58:32,804 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-04 07:58:32,805 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-04 07:58:32,805 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-04 07:58:32,805 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-04 07:58:32,806 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-04 07:58:32,806 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-04 07:58:32,806 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-04 07:58:32,809 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-04 07:58:32,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-04 07:58:32,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-04 07:58:32,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-04 07:58:32,822 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-04 07:58:32,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-04 07:58:32,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-04 07:58:32,825 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:32,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-04 07:58:32,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-04 07:58:32,827 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-04 07:58:32,829 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 07:58:32,829 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:32,829 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 07:58:32,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33811,1693814312612, sessionid=0x10075f2862e0000, setting cluster-up flag (Was=false) 2023-09-04 07:58:32,837 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:32,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-04 07:58:32,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:58:32,846 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:32,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-04 07:58:32,852 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:58:32,853 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.hbase-snapshot/.tmp 2023-09-04 07:58:32,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-04 07:58:32,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:58:32,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:58:32,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:58:32,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:58:32,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-04 07:58:32,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 07:58:32,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693814342858 2023-09-04 07:58:32,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-04 07:58:32,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-04 07:58:32,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-04 07:58:32,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-04 07:58:32,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-04 07:58:32,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-04 07:58:32,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,860 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-04 07:58:32,860 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-04 07:58:32,860 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 07:58:32,860 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-04 07:58:32,861 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-04 07:58:32,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-04 07:58:32,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-04 07:58:32,861 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814312861,5,FailOnTimeoutGroup] 2023-09-04 07:58:32,861 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814312861,5,FailOnTimeoutGroup] 2023-09-04 07:58:32,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-04 07:58:32,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,862 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 07:58:32,879 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 07:58:32,880 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 07:58:32,881 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc 2023-09-04 07:58:32,892 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(951): ClusterId : 4e03d1df-b776-47c0-bef2-59ac9d80669c 2023-09-04 07:58:32,893 DEBUG [RS:0;jenkins-hbase4:35915] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-04 07:58:32,894 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:32,896 DEBUG [RS:0;jenkins-hbase4:35915] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-04 07:58:32,896 DEBUG [RS:0;jenkins-hbase4:35915] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-04 07:58:32,896 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 07:58:32,898 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/info 2023-09-04 07:58:32,899 DEBUG [RS:0;jenkins-hbase4:35915] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-04 07:58:32,899 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 07:58:32,900 DEBUG [RS:0;jenkins-hbase4:35915] zookeeper.ReadOnlyZKClient(139): Connect 0x15956188 to 127.0.0.1:49378 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:58:32,901 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:32,902 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 07:58:32,904 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:58:32,905 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 07:58:32,905 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:32,906 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 07:58:32,907 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/table 2023-09-04 07:58:32,908 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 07:58:32,908 DEBUG [RS:0;jenkins-hbase4:35915] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b528d4a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:58:32,909 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:32,909 DEBUG [RS:0;jenkins-hbase4:35915] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f6d0af8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 07:58:32,910 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740 2023-09-04 07:58:32,911 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740 2023-09-04 07:58:32,914 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 07:58:32,916 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 07:58:32,918 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:58:32,919 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=823001, jitterRate=0.046500593423843384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 07:58:32,920 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 07:58:32,920 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:35915 2023-09-04 07:58:32,920 INFO [RS:0;jenkins-hbase4:35915] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-04 07:58:32,920 INFO [RS:0;jenkins-hbase4:35915] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-04 07:58:32,920 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1022): About to register with Master. 2023-09-04 07:58:32,920 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 07:58:32,920 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 07:58:32,920 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 07:58:32,920 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 07:58:32,920 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 07:58:32,920 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33811,1693814312612 with isa=jenkins-hbase4.apache.org/172.31.14.131:35915, startcode=1693814312663 2023-09-04 07:58:32,921 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 07:58:32,921 DEBUG [RS:0;jenkins-hbase4:35915] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-04 07:58:32,921 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 07:58:32,923 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 07:58:32,923 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-04 07:58:32,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-04 07:58:32,924 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49345, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-09-04 07:58:32,925 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33811] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:32,925 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc 2023-09-04 07:58:32,925 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39673 2023-09-04 07:58:32,926 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-04 07:58:32,926 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-04 07:58:32,928 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-04 07:58:32,928 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 07:58:32,929 DEBUG [RS:0;jenkins-hbase4:35915] zookeeper.ZKUtil(162): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:32,929 WARN [RS:0;jenkins-hbase4:35915] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 07:58:32,929 INFO [RS:0;jenkins-hbase4:35915] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:58:32,929 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:32,930 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35915,1693814312663] 2023-09-04 07:58:32,933 DEBUG [RS:0;jenkins-hbase4:35915] zookeeper.ZKUtil(162): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:32,934 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-04 07:58:32,934 INFO [RS:0;jenkins-hbase4:35915] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-04 07:58:32,937 INFO [RS:0;jenkins-hbase4:35915] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-04 07:58:32,937 INFO [RS:0;jenkins-hbase4:35915] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 07:58:32,937 INFO [RS:0;jenkins-hbase4:35915] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,937 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-04 07:58:32,938 INFO [RS:0;jenkins-hbase4:35915] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,939 DEBUG [RS:0;jenkins-hbase4:35915] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:58:32,941 INFO [RS:0;jenkins-hbase4:35915] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,941 INFO [RS:0;jenkins-hbase4:35915] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,941 INFO [RS:0;jenkins-hbase4:35915] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,955 INFO [RS:0;jenkins-hbase4:35915] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-04 07:58:32,955 INFO [RS:0;jenkins-hbase4:35915] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35915,1693814312663-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:32,966 INFO [RS:0;jenkins-hbase4:35915] regionserver.Replication(203): jenkins-hbase4.apache.org,35915,1693814312663 started 2023-09-04 07:58:32,967 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,35915,1693814312663, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35915, sessionid=0x10075f2862e0001 2023-09-04 07:58:32,967 DEBUG [RS:0;jenkins-hbase4:35915] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-04 07:58:32,967 DEBUG [RS:0;jenkins-hbase4:35915] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:32,967 DEBUG [RS:0;jenkins-hbase4:35915] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35915,1693814312663' 2023-09-04 07:58:32,967 DEBUG [RS:0;jenkins-hbase4:35915] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:58:32,967 DEBUG [RS:0;jenkins-hbase4:35915] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:58:32,968 DEBUG [RS:0;jenkins-hbase4:35915] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-04 07:58:32,968 DEBUG [RS:0;jenkins-hbase4:35915] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-04 07:58:32,968 DEBUG [RS:0;jenkins-hbase4:35915] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:32,968 DEBUG [RS:0;jenkins-hbase4:35915] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35915,1693814312663' 2023-09-04 07:58:32,968 DEBUG [RS:0;jenkins-hbase4:35915] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-04 07:58:32,968 DEBUG [RS:0;jenkins-hbase4:35915] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-04 07:58:32,968 DEBUG [RS:0;jenkins-hbase4:35915] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-04 07:58:32,968 INFO [RS:0;jenkins-hbase4:35915] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-04 07:58:32,969 INFO [RS:0;jenkins-hbase4:35915] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-04 07:58:33,071 INFO [RS:0;jenkins-hbase4:35915] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35915%2C1693814312663, suffix=, logDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663, archiveDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/oldWALs, maxLogs=32 2023-09-04 07:58:33,078 DEBUG [jenkins-hbase4:33811] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-04 07:58:33,079 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35915,1693814312663, state=OPENING 2023-09-04 07:58:33,081 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-04 07:58:33,082 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:33,083 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 07:58:33,083 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35915,1693814312663}] 2023-09-04 07:58:33,092 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK] 2023-09-04 07:58:33,092 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK] 2023-09-04 07:58:33,094 INFO [RS:0;jenkins-hbase4:35915] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814313073 2023-09-04 07:58:33,095 DEBUG [RS:0;jenkins-hbase4:35915] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK], DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK]] 2023-09-04 07:58:33,239 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:33,239 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-04 07:58:33,241 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52806, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-04 07:58:33,292 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-04 07:58:33,292 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:58:33,294 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35915%2C1693814312663.meta, suffix=.meta, logDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663, archiveDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/oldWALs, maxLogs=32 2023-09-04 07:58:33,311 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK] 2023-09-04 07:58:33,313 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK] 2023-09-04 07:58:33,315 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.meta.1693814313295.meta 2023-09-04 07:58:33,315 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK], DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK]] 2023-09-04 07:58:33,315 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:58:33,316 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-04 07:58:33,316 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-04 07:58:33,316 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-04 07:58:33,316 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-04 07:58:33,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:33,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-04 07:58:33,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-04 07:58:33,319 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 07:58:33,320 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/info 2023-09-04 07:58:33,320 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/info 2023-09-04 07:58:33,321 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 07:58:33,321 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:33,322 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 07:58:33,323 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:58:33,323 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:58:33,323 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 07:58:33,324 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:33,324 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 07:58:33,325 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/table 2023-09-04 07:58:33,325 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/table 2023-09-04 07:58:33,327 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 07:58:33,327 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:33,328 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740 2023-09-04 07:58:33,330 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740 2023-09-04 07:58:33,332 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 07:58:33,334 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 07:58:33,335 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=758234, jitterRate=-0.03585684299468994}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 07:58:33,335 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 07:58:33,336 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693814313239 2023-09-04 07:58:33,340 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-04 07:58:33,340 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-04 07:58:33,341 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35915,1693814312663, state=OPEN 2023-09-04 07:58:33,343 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-04 07:58:33,343 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 07:58:33,345 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-04 07:58:33,346 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35915,1693814312663 in 260 msec 2023-09-04 07:58:33,348 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-04 07:58:33,348 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 423 msec 2023-09-04 07:58:33,351 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 495 msec 2023-09-04 07:58:33,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693814313351, completionTime=-1 2023-09-04 07:58:33,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-04 07:58:33,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-04 07:58:33,353 DEBUG [hconnection-0x2092c607-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 07:58:33,355 INFO [RS-EventLoopGroup-6-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52816, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 07:58:33,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-04 07:58:33,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693814373357 2023-09-04 07:58:33,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693814433357 2023-09-04 07:58:33,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-09-04 07:58:33,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33811,1693814312612-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:33,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33811,1693814312612-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:33,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33811,1693814312612-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:33,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33811, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:33,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-04 07:58:33,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-04 07:58:33,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 07:58:33,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-04 07:58:33,366 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-04 07:58:33,368 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 07:58:33,369 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 07:58:33,371 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,371 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291 empty. 2023-09-04 07:58:33,372 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,372 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-04 07:58:33,389 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-04 07:58:33,390 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 07ec41a55ef66ec12093c83fd250c291, NAME => 'hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp 2023-09-04 07:58:33,400 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:33,400 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 07ec41a55ef66ec12093c83fd250c291, disabling compactions & flushes 2023-09-04 07:58:33,401 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:33,401 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:33,401 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. after waiting 0 ms 2023-09-04 07:58:33,401 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:33,401 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:33,401 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 07ec41a55ef66ec12093c83fd250c291: 2023-09-04 07:58:33,403 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 07:58:33,405 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814313405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814313405"}]},"ts":"1693814313405"} 2023-09-04 07:58:33,407 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 07:58:33,408 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 07:58:33,408 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814313408"}]},"ts":"1693814313408"} 2023-09-04 07:58:33,410 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-04 07:58:33,418 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=07ec41a55ef66ec12093c83fd250c291, ASSIGN}] 2023-09-04 07:58:33,420 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=07ec41a55ef66ec12093c83fd250c291, ASSIGN 2023-09-04 07:58:33,421 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=07ec41a55ef66ec12093c83fd250c291, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35915,1693814312663; forceNewPlan=false, retain=false 2023-09-04 07:58:33,572 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=07ec41a55ef66ec12093c83fd250c291, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:33,572 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814313572"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814313572"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814313572"}]},"ts":"1693814313572"} 2023-09-04 07:58:33,575 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 07ec41a55ef66ec12093c83fd250c291, server=jenkins-hbase4.apache.org,35915,1693814312663}] 2023-09-04 07:58:33,733 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:33,733 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 07ec41a55ef66ec12093c83fd250c291, NAME => 'hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:58:33,733 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,734 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:33,734 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,734 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,735 INFO [StoreOpener-07ec41a55ef66ec12093c83fd250c291-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,737 DEBUG [StoreOpener-07ec41a55ef66ec12093c83fd250c291-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291/info 2023-09-04 07:58:33,737 DEBUG [StoreOpener-07ec41a55ef66ec12093c83fd250c291-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291/info 2023-09-04 07:58:33,737 INFO [StoreOpener-07ec41a55ef66ec12093c83fd250c291-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 07ec41a55ef66ec12093c83fd250c291 columnFamilyName info 2023-09-04 07:58:33,738 INFO [StoreOpener-07ec41a55ef66ec12093c83fd250c291-1] regionserver.HStore(310): Store=07ec41a55ef66ec12093c83fd250c291/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:33,739 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,740 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,743 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:58:33,745 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:58:33,746 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 07ec41a55ef66ec12093c83fd250c291; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=751330, jitterRate=-0.044635385274887085}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:58:33,746 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 07ec41a55ef66ec12093c83fd250c291: 2023-09-04 07:58:33,747 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291., pid=6, masterSystemTime=1693814313728 2023-09-04 07:58:33,750 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:33,750 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:33,750 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=07ec41a55ef66ec12093c83fd250c291, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:33,751 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814313750"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814313750"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814313750"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814313750"}]},"ts":"1693814313750"} 2023-09-04 07:58:33,756 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-04 07:58:33,756 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 07ec41a55ef66ec12093c83fd250c291, server=jenkins-hbase4.apache.org,35915,1693814312663 in 178 msec 2023-09-04 07:58:33,759 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-04 07:58:33,759 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=07ec41a55ef66ec12093c83fd250c291, ASSIGN in 338 msec 2023-09-04 07:58:33,760 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 07:58:33,760 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814313760"}]},"ts":"1693814313760"} 2023-09-04 07:58:33,762 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-04 07:58:33,764 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 07:58:33,766 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 399 msec 2023-09-04 07:58:33,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-04 07:58:33,768 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:58:33,768 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:33,773 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-04 07:58:33,788 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:58:33,795 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 21 msec 2023-09-04 07:58:33,805 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-04 07:58:33,814 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:58:33,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 12 msec 2023-09-04 07:58:33,830 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-04 07:58:33,832 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-04 07:58:33,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.134sec 2023-09-04 07:58:33,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-04 07:58:33,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-04 07:58:33,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-04 07:58:33,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33811,1693814312612-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-04 07:58:33,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33811,1693814312612-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-04 07:58:33,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-04 07:58:33,891 DEBUG [Listener at localhost/44221] zookeeper.ReadOnlyZKClient(139): Connect 0x2f4bd615 to 127.0.0.1:49378 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:58:33,897 DEBUG [Listener at localhost/44221] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@72048674, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:58:33,899 DEBUG [hconnection-0x537c86b2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 07:58:33,901 INFO [RS-EventLoopGroup-6-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52820, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 07:58:33,903 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:58:33,903 INFO [Listener at localhost/44221] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:58:33,907 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-04 07:58:33,907 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:58:33,908 INFO [Listener at localhost/44221] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-04 07:58:33,910 DEBUG [Listener at localhost/44221] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-04 07:58:33,914 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52524, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-04 07:58:33,916 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-04 07:58:33,916 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-04 07:58:33,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 07:58:33,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:33,920 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 07:58:33,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testCompactionRecordDoesntBlockRolling" procId is: 9 2023-09-04 07:58:33,922 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 07:58:33,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 07:58:33,925 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:33,925 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9 empty. 2023-09-04 07:58:33,926 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:33,926 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testCompactionRecordDoesntBlockRolling regions 2023-09-04 07:58:33,941 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/.tabledesc/.tableinfo.0000000001 2023-09-04 07:58:33,943 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => e85e122dc7ac3f058aa24020708ecde9, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testCompactionRecordDoesntBlockRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/.tmp 2023-09-04 07:58:33,956 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:33,956 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1604): Closing e85e122dc7ac3f058aa24020708ecde9, disabling compactions & flushes 2023-09-04 07:58:33,956 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:33,956 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:33,957 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. after waiting 0 ms 2023-09-04 07:58:33,957 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:33,957 INFO [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:33,957 DEBUG [RegionOpenAndInit-TestLogRolling-testCompactionRecordDoesntBlockRolling-pool-0] regionserver.HRegion(1558): Region close journal for e85e122dc7ac3f058aa24020708ecde9: 2023-09-04 07:58:33,960 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 07:58:33,961 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693814313961"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814313961"}]},"ts":"1693814313961"} 2023-09-04 07:58:33,963 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 07:58:33,964 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 07:58:33,965 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814313964"}]},"ts":"1693814313964"} 2023-09-04 07:58:33,967 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLING in hbase:meta 2023-09-04 07:58:33,970 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=e85e122dc7ac3f058aa24020708ecde9, ASSIGN}] 2023-09-04 07:58:33,972 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=e85e122dc7ac3f058aa24020708ecde9, ASSIGN 2023-09-04 07:58:33,973 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=e85e122dc7ac3f058aa24020708ecde9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35915,1693814312663; forceNewPlan=false, retain=false 2023-09-04 07:58:34,124 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e85e122dc7ac3f058aa24020708ecde9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:34,124 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693814314124"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814314124"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814314124"}]},"ts":"1693814314124"} 2023-09-04 07:58:34,127 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure e85e122dc7ac3f058aa24020708ecde9, server=jenkins-hbase4.apache.org,35915,1693814312663}] 2023-09-04 07:58:34,183 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 07:58:34,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:34,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e85e122dc7ac3f058aa24020708ecde9, NAME => 'TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:58:34,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testCompactionRecordDoesntBlockRolling e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:34,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:58:34,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:34,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:34,287 INFO [StoreOpener-e85e122dc7ac3f058aa24020708ecde9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:34,290 DEBUG [StoreOpener-e85e122dc7ac3f058aa24020708ecde9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info 2023-09-04 07:58:34,290 DEBUG [StoreOpener-e85e122dc7ac3f058aa24020708ecde9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info 2023-09-04 07:58:34,290 INFO [StoreOpener-e85e122dc7ac3f058aa24020708ecde9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e85e122dc7ac3f058aa24020708ecde9 columnFamilyName info 2023-09-04 07:58:34,291 INFO [StoreOpener-e85e122dc7ac3f058aa24020708ecde9-1] regionserver.HStore(310): Store=e85e122dc7ac3f058aa24020708ecde9/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:58:34,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:34,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:34,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:58:34,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:58:34,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e85e122dc7ac3f058aa24020708ecde9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=737405, jitterRate=-0.06234206259250641}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:58:34,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e85e122dc7ac3f058aa24020708ecde9: 2023-09-04 07:58:34,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9., pid=11, masterSystemTime=1693814314280 2023-09-04 07:58:34,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:34,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:34,303 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e85e122dc7ac3f058aa24020708ecde9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:34,304 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.","families":{"info":[{"qualifier":"regioninfo","vlen":87,"tag":[],"timestamp":"1693814314303"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814314303"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814314303"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814314303"}]},"ts":"1693814314303"} 2023-09-04 07:58:34,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-04 07:58:34,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure e85e122dc7ac3f058aa24020708ecde9, server=jenkins-hbase4.apache.org,35915,1693814312663 in 179 msec 2023-09-04 07:58:34,313 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-04 07:58:34,313 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling, region=e85e122dc7ac3f058aa24020708ecde9, ASSIGN in 339 msec 2023-09-04 07:58:34,314 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 07:58:34,315 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testCompactionRecordDoesntBlockRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814314314"}]},"ts":"1693814314314"} 2023-09-04 07:58:34,316 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testCompactionRecordDoesntBlockRolling, state=ENABLED in hbase:meta 2023-09-04 07:58:34,319 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 07:58:34,321 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testCompactionRecordDoesntBlockRolling in 403 msec 2023-09-04 07:58:38,146 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 07:58:38,147 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-09-04 07:58:39,237 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 07:58:39,264 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-04 07:58:39,264 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-04 07:58:39,265 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:58:43,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 07:58:43,924 INFO [Listener at localhost/44221] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testCompactionRecordDoesntBlockRolling, procId: 9 completed 2023-09-04 07:58:43,927 DEBUG [Listener at localhost/44221] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:43,927 DEBUG [Listener at localhost/44221] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:43,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 07:58:43,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure.ProcedureCoordinator(165): Submitting procedure hbase:namespace 2023-09-04 07:58:43,948 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'hbase:namespace' 2023-09-04 07:58:43,948 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:58:43,948 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'hbase:namespace' starting 'acquire' 2023-09-04 07:58:43,948 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'hbase:namespace', kicking off acquire phase on members. 2023-09-04 07:58:43,949 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:43,949 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 07:58:43,950 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:58:43,950 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:58:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:58:43,951 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,951 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 07:58:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 07:58:43,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 07:58:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 07:58:43,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for hbase:namespace 2023-09-04 07:58:43,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:hbase:namespace 2023-09-04 07:58:43,954 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'hbase:namespace' with timeout 60000ms 2023-09-04 07:58:43,954 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:58:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'hbase:namespace' starting 'acquire' stage 2023-09-04 07:58:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 07:58:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 07:58:43,955 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:43,955 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. started... 2023-09-04 07:58:43,956 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 07ec41a55ef66ec12093c83fd250c291 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-04 07:58:43,970 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291/.tmp/info/d0abe34292ae4f3da6c83faccdbf27fd 2023-09-04 07:58:43,979 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291/.tmp/info/d0abe34292ae4f3da6c83faccdbf27fd as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291/info/d0abe34292ae4f3da6c83faccdbf27fd 2023-09-04 07:58:43,986 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291/info/d0abe34292ae4f3da6c83faccdbf27fd, entries=2, sequenceid=6, filesize=4.8 K 2023-09-04 07:58:43,987 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 07ec41a55ef66ec12093c83fd250c291 in 31ms, sequenceid=6, compaction requested=false 2023-09-04 07:58:43,988 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 07ec41a55ef66ec12093c83fd250c291: 2023-09-04 07:58:43,988 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:58:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 07:58:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 07:58:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'hbase:namespace' locally acquired 2023-09-04 07:58:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure (hbase:namespace) in zk 2023-09-04 07:58:43,991 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,991 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 07:58:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:58:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:58:43,992 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 07:58:43,992 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'hbase:namespace' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 07:58:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:58:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:58:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 07:58:43,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:58:43,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure 'hbase:namespace' on coordinator 2023-09-04 07:58:43,993 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'hbase:namespace' starting 'in-barrier' execution. 2023-09-04 07:58:43,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22a16139[Count = 0] remaining members to acquire global barrier 2023-09-04 07:58:43,993 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 07:58:43,996 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 07:58:43,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 07:58:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 07:58:43,996 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'hbase:namespace' received 'reached' from coordinator. 2023-09-04 07:58:43,996 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'hbase:namespace' locally completed 2023-09-04 07:58:43,996 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'hbase:namespace' completed for member 'jenkins-hbase4.apache.org,35915,1693814312663' in zk 2023-09-04 07:58:43,996 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,996 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 07:58:43,997 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'hbase:namespace' has notified controller of completion 2023-09-04 07:58:43,997 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,998 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:58:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:58:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:58:43,998 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'hbase:namespace' completed. 2023-09-04 07:58:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:58:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:58:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 07:58:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:58:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 07:58:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'hbase:namespace' member 'jenkins-hbase4.apache.org,35915,1693814312663': 2023-09-04 07:58:44,001 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'hbase:namespace' execution completed 2023-09-04 07:58:44,001 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 07:58:44,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' released barrier for procedure'hbase:namespace', counting down latch. Waiting for 0 more 2023-09-04 07:58:44,001 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 07:58:44,001 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:hbase:namespace 2023-09-04 07:58:44,001 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure hbase:namespaceincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 07:58:44,003 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:44,003 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:44,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:44,003 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:58:44,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:44,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:58:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:58:44,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:58:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:58:44,004 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:58:44,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 07:58:44,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:44,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:58:44,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 07:58:44,006 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:44,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:44,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:58:44,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----hbase:namespace 2023-09-04 07:58:44,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/hbase:namespace 2023-09-04 07:58:44,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 07:58:44,013 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/hbase:namespace 2023-09-04 07:58:44,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'hbase:namespace' 2023-09-04 07:58:44,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:58:44,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 07:58:44,013 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/hbase:namespace 2023-09-04 07:58:44,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:58:44,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:58:44,016 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : hbase:namespace'' to complete. (max 20000 ms per retry) 2023-09-04 07:58:44,016 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 07:58:54,016 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 07:58:54,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 07:58:54,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 07:58:54,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,034 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:58:54,034 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:58:54,035 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-04 07:58:54,035 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-04 07:58:54,035 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,035 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,038 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,038 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:58:54,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:58:54,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:58:54,038 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,039 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 07:58:54,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 07:58:54,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,039 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-04 07:58:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:58:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-04 07:58:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 07:58:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 07:58:54,040 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:54,040 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. started... 2023-09-04 07:58:54,041 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e85e122dc7ac3f058aa24020708ecde9 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 07:58:54,056 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=5 (bloomFilter=true), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/1d981496b1c24544af4ef3bd0cc6a8b9 2023-09-04 07:58:54,066 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/1d981496b1c24544af4ef3bd0cc6a8b9 as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/1d981496b1c24544af4ef3bd0cc6a8b9 2023-09-04 07:58:54,072 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/1d981496b1c24544af4ef3bd0cc6a8b9, entries=1, sequenceid=5, filesize=5.8 K 2023-09-04 07:58:54,073 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e85e122dc7ac3f058aa24020708ecde9 in 32ms, sequenceid=5, compaction requested=false 2023-09-04 07:58:54,074 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e85e122dc7ac3f058aa24020708ecde9: 2023-09-04 07:58:54,074 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:58:54,074 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 07:58:54,074 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 07:58:54,074 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,074 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-04 07:58:54,074 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-04 07:58:54,076 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,076 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:58:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:58:54,077 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,077 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 07:58:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:58:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:58:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:58:54,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-04 07:58:54,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@269ee1b5[Count = 0] remaining members to acquire global barrier 2023-09-04 07:58:54,079 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-04 07:58:54,079 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,081 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,081 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-04 07:58:54,081 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-04 07:58:54,081 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,35915,1693814312663' in zk 2023-09-04 07:58:54,081 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,081 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 07:58:54,083 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,083 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-04 07:58:54,083 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:58:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:58:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:58:54,083 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-04 07:58:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:58:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:58:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:58:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,35915,1693814312663': 2023-09-04 07:58:54,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-04 07:58:54,086 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-04 07:58:54,086 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 07:58:54,086 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 07:58:54,086 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,086 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 07:58:54,097 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,097 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:58:54,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:58:54,097 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:58:54,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:58:54,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:58:54,098 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:58:54,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:58:54,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,099 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:58:54,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,102 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,102 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:58:54,102 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:58:54,102 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-04 07:58:54,102 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:58:54,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 07:58:54,102 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:58:54,102 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:58:54,102 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:58:54,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:58:54,103 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,103 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,103 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:58:54,103 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-04 07:58:54,103 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 07:58:54,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:58:54,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:59:04,103 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 07:59:04,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 07:59:04,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 07:59:04,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-04 07:59:04,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,114 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:59:04,114 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:59:04,115 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-04 07:59:04,115 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-04 07:59:04,115 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,116 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,117 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:59:04,117 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:59:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:59:04,117 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,117 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 07:59:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 07:59:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,118 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-09-04 07:59:04,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,119 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-04 07:59:04,119 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:59:04,119 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-04 07:59:04,120 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 07:59:04,120 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 07:59:04,120 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:04,120 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. started... 2023-09-04 07:59:04,120 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e85e122dc7ac3f058aa24020708ecde9 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 07:59:04,132 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=9 (bloomFilter=true), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/9ad7b684b27d4472bf7cef4697d427ce 2023-09-04 07:59:04,140 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/9ad7b684b27d4472bf7cef4697d427ce as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/9ad7b684b27d4472bf7cef4697d427ce 2023-09-04 07:59:04,145 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/9ad7b684b27d4472bf7cef4697d427ce, entries=1, sequenceid=9, filesize=5.8 K 2023-09-04 07:59:04,146 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e85e122dc7ac3f058aa24020708ecde9 in 26ms, sequenceid=9, compaction requested=false 2023-09-04 07:59:04,146 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e85e122dc7ac3f058aa24020708ecde9: 2023-09-04 07:59:04,146 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:04,146 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 07:59:04,147 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 07:59:04,147 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,147 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-04 07:59:04,147 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-04 07:59:04,149 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,149 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:04,149 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,149 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 07:59:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:04,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-04 07:59:04,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@639c6b02[Count = 0] remaining members to acquire global barrier 2023-09-04 07:59:04,151 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-04 07:59:04,151 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,155 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-04 07:59:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-04 07:59:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,35915,1693814312663' in zk 2023-09-04 07:59:04,156 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,156 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 07:59:04,157 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-04 07:59:04,157 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,157 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:59:04,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:04,157 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-04 07:59:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:04,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,35915,1693814312663': 2023-09-04 07:59:04,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-04 07:59:04,160 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-04 07:59:04,160 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 07:59:04,160 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 07:59:04,160 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,160 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 07:59:04,162 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,162 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,162 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:59:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:04,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:59:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:59:04,163 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,164 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,166 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,166 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,166 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:59:04,166 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,167 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:59:04,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-04 07:59:04,167 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:04,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:59:04,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 07:59:04,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:59:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:59:04,166 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:59:04,167 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,167 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,167 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:04,167 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-04 07:59:04,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:59:04,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:59:04,168 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 07:59:14,168 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 07:59:14,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 07:59:14,188 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK] 2023-09-04 07:59:14,188 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK] 2023-09-04 07:59:14,191 INFO [Listener at localhost/44221] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814313073 with entries=13, filesize=6.36 KB; new WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814354172 2023-09-04 07:59:14,191 DEBUG [Listener at localhost/44221] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK], DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK]] 2023-09-04 07:59:14,191 DEBUG [Listener at localhost/44221] wal.AbstractFSWAL(716): hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814313073 is not closed yet, will try archiving it next time 2023-09-04 07:59:14,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 07:59:14,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-04 07:59:14,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,201 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:59:14,201 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:59:14,201 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-04 07:59:14,201 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-04 07:59:14,202 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,202 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,207 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,207 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:59:14,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:59:14,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:59:14,207 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,207 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 07:59:14,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 07:59:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,208 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,208 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-09-04 07:59:14,208 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,208 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-04 07:59:14,208 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:59:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-04 07:59:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 07:59:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 07:59:14,209 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:14,209 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. started... 2023-09-04 07:59:14,209 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e85e122dc7ac3f058aa24020708ecde9 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 07:59:14,220 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=13 (bloomFilter=true), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/3fcd969e805544a3a04b220bbb7024e8 2023-09-04 07:59:14,229 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/3fcd969e805544a3a04b220bbb7024e8 as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/3fcd969e805544a3a04b220bbb7024e8 2023-09-04 07:59:14,236 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/3fcd969e805544a3a04b220bbb7024e8, entries=1, sequenceid=13, filesize=5.8 K 2023-09-04 07:59:14,237 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e85e122dc7ac3f058aa24020708ecde9 in 28ms, sequenceid=13, compaction requested=true 2023-09-04 07:59:14,237 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e85e122dc7ac3f058aa24020708ecde9: 2023-09-04 07:59:14,237 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:14,237 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 07:59:14,237 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 07:59:14,237 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,237 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-04 07:59:14,237 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-04 07:59:14,239 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,239 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:14,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:14,239 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,239 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 07:59:14,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:14,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:14,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:14,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-04 07:59:14,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b34f516[Count = 0] remaining members to acquire global barrier 2023-09-04 07:59:14,241 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-04 07:59:14,241 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,242 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,243 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-04 07:59:14,243 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-04 07:59:14,243 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,35915,1693814312663' in zk 2023-09-04 07:59:14,243 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,243 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 07:59:14,244 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,244 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-04 07:59:14,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:14,244 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:59:14,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:14,244 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-04 07:59:14,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:14,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:14,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,35915,1693814312663': 2023-09-04 07:59:14,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-04 07:59:14,247 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-04 07:59:14,248 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 07:59:14,248 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 07:59:14,248 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,248 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 07:59:14,250 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,250 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:14,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:14,250 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:59:14,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:14,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:59:14,251 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:59:14,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:14,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,252 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:14,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,255 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,255 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:59:14,255 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,255 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:59:14,255 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:59:14,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 07:59:14,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:59:14,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:59:14,255 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:59:14,255 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:14,256 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,256 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,256 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:14,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:59:14,256 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-04 07:59:14,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:59:14,256 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 07:59:24,256 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 07:59:24,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 07:59:24,259 DEBUG [Listener at localhost/44221] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 07:59:24,264 DEBUG [Listener at localhost/44221] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 17694 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 07:59:24,266 DEBUG [Listener at localhost/44221] regionserver.HStore(1912): e85e122dc7ac3f058aa24020708ecde9/info is initiating minor compaction (all files) 2023-09-04 07:59:24,266 INFO [Listener at localhost/44221] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 07:59:24,266 INFO [Listener at localhost/44221] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:24,267 INFO [Listener at localhost/44221] regionserver.HRegion(2259): Starting compaction of e85e122dc7ac3f058aa24020708ecde9/info in TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:24,267 INFO [Listener at localhost/44221] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/1d981496b1c24544af4ef3bd0cc6a8b9, hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/9ad7b684b27d4472bf7cef4697d427ce, hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/3fcd969e805544a3a04b220bbb7024e8] into tmpdir=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp, totalSize=17.3 K 2023-09-04 07:59:24,268 DEBUG [Listener at localhost/44221] compactions.Compactor(207): Compacting 1d981496b1c24544af4ef3bd0cc6a8b9, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1693814334026 2023-09-04 07:59:24,268 DEBUG [Listener at localhost/44221] compactions.Compactor(207): Compacting 9ad7b684b27d4472bf7cef4697d427ce, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1693814344106 2023-09-04 07:59:24,269 DEBUG [Listener at localhost/44221] compactions.Compactor(207): Compacting 3fcd969e805544a3a04b220bbb7024e8, keycount=1, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1693814354170 2023-09-04 07:59:24,293 INFO [Listener at localhost/44221] throttle.PressureAwareThroughputController(145): e85e122dc7ac3f058aa24020708ecde9#info#compaction#9 average throughput is 3.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 07:59:24,314 DEBUG [Listener at localhost/44221] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/8202055534234b6e9007689952b7bf59 as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/8202055534234b6e9007689952b7bf59 2023-09-04 07:59:24,328 INFO [Listener at localhost/44221] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in e85e122dc7ac3f058aa24020708ecde9/info of e85e122dc7ac3f058aa24020708ecde9 into 8202055534234b6e9007689952b7bf59(size=7.9 K), total size for store is 7.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 07:59:24,328 DEBUG [Listener at localhost/44221] regionserver.HRegion(2289): Compaction status journal for e85e122dc7ac3f058aa24020708ecde9: 2023-09-04 07:59:24,344 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK] 2023-09-04 07:59:24,345 DEBUG [RS-EventLoopGroup-6-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK] 2023-09-04 07:59:24,346 INFO [Listener at localhost/44221] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814354172 with entries=4, filesize=2.43 KB; new WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814364330 2023-09-04 07:59:24,347 DEBUG [Listener at localhost/44221] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK], DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK]] 2023-09-04 07:59:24,347 DEBUG [Listener at localhost/44221] wal.AbstractFSWAL(716): hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814354172 is not closed yet, will try archiving it next time 2023-09-04 07:59:24,347 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814313073 to hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/oldWALs/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814313073 2023-09-04 07:59:24,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-09-04 07:59:24,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure.ProcedureCoordinator(143): Procedure TestLogRolling-testCompactionRecordDoesntBlockRolling was in running list but was completed. Accepting new attempt. 2023-09-04 07:59:24,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] procedure.ProcedureCoordinator(165): Submitting procedure TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,355 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:59:24,355 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:59:24,355 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' 2023-09-04 07:59:24,355 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling', kicking off acquire phase on members. 2023-09-04 07:59:24,356 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,356 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,360 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,360 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:59:24,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:59:24,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:59:24,360 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,360 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-09-04 07:59:24,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-09-04 07:59:24,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc TestLogRolling-testCompactionRecordDoesntBlockRolling is still present, removing 2023-09-04 07:59:24,362 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' with timeout 60000ms 2023-09-04 07:59:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-09-04 07:59:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'acquire' stage 2023-09-04 07:59:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-09-04 07:59:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-09-04 07:59:24,362 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:24,362 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. started... 2023-09-04 07:59:24,363 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e85e122dc7ac3f058aa24020708ecde9 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 07:59:24,379 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=18 (bloomFilter=true), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/75c3233b1f52425ebdb5356087bb32c8 2023-09-04 07:59:24,387 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/75c3233b1f52425ebdb5356087bb32c8 as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/75c3233b1f52425ebdb5356087bb32c8 2023-09-04 07:59:24,393 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/75c3233b1f52425ebdb5356087bb32c8, entries=1, sequenceid=18, filesize=5.8 K 2023-09-04 07:59:24,394 INFO [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e85e122dc7ac3f058aa24020708ecde9 in 31ms, sequenceid=18, compaction requested=false 2023-09-04 07:59:24,394 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e85e122dc7ac3f058aa24020708ecde9: 2023-09-04 07:59:24,394 DEBUG [rs(jenkins-hbase4.apache.org,35915,1693814312663)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:24,394 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-09-04 07:59:24,395 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-09-04 07:59:24,395 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,395 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally acquired 2023-09-04 07:59:24,395 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure (TestLogRolling-testCompactionRecordDoesntBlockRolling) in zk 2023-09-04 07:59:24,397 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,397 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:24,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:24,398 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,398 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-09-04 07:59:24,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:24,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:24,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:24,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35915,1693814312663' joining acquired barrier for procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' on coordinator 2023-09-04 07:59:24,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52b5ebe0[Count = 0] remaining members to acquire global barrier 2023-09-04 07:59:24,399 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' starting 'in-barrier' execution. 2023-09-04 07:59:24,399 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,401 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,401 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' received 'reached' from coordinator. 2023-09-04 07:59:24,401 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' locally completed 2023-09-04 07:59:24,401 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed for member 'jenkins-hbase4.apache.org,35915,1693814312663' in zk 2023-09-04 07:59:24,401 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,401 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-09-04 07:59:24,402 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' has notified controller of completion 2023-09-04 07:59:24,402 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,402 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:59:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:24,402 DEBUG [member: 'jenkins-hbase4.apache.org,35915,1693814312663' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' completed. 2023-09-04 07:59:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:24,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:24,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' member 'jenkins-hbase4.apache.org,35915,1693814312663': 2023-09-04 07:59:24,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35915,1693814312663' released barrier for procedure'TestLogRolling-testCompactionRecordDoesntBlockRolling', counting down latch. Waiting for 0 more 2023-09-04 07:59:24,406 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'TestLogRolling-testCompactionRecordDoesntBlockRolling' execution completed 2023-09-04 07:59:24,406 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-09-04 07:59:24,406 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-09-04 07:59:24,406 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,406 INFO [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure TestLogRolling-testCompactionRecordDoesntBlockRollingincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-09-04 07:59:24,408 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,408 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-09-04 07:59:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-09-04 07:59:24,408 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:59:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-09-04 07:59:24,408 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:59:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:59:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-09-04 07:59:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,410 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-09-04 07:59:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,413 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling 2023-09-04 07:59:24,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-09-04 07:59:24,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'TestLogRolling-testCompactionRecordDoesntBlockRolling' 2023-09-04 07:59:24,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-09-04 07:59:24,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/TestLogRolling-testCompactionRecordDoesntBlockRolling because node does not exist (not an error) 2023-09-04 07:59:24,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:59:24,413 DEBUG [(jenkins-hbase4.apache.org,33811,1693814312612)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-09-04 07:59:24,414 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : TestLogRolling-testCompactionRecordDoesntBlockRolling'' to complete. (max 20000 ms per retry) 2023-09-04 07:59:24,415 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2698): (#1) Sleeping: 10000ms while waiting for procedure completion. 2023-09-04 07:59:24,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-09-04 07:59:24,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:59:34,415 DEBUG [Listener at localhost/44221] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-09-04 07:59:34,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33811] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-09-04 07:59:34,434 DEBUG [RS-EventLoopGroup-6-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK] 2023-09-04 07:59:34,434 DEBUG [RS-EventLoopGroup-6-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK] 2023-09-04 07:59:34,436 INFO [Listener at localhost/44221] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814364330 with entries=3, filesize=1.94 KB; new WAL /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814374419 2023-09-04 07:59:34,436 DEBUG [Listener at localhost/44221] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33895,DS-fdacbefd-062a-4c61-b6db-a60e00e59a7e,DISK], DatanodeInfoWithStorage[127.0.0.1:34853,DS-e2ecdc25-0802-496e-ad7d-59a19951d5db,DISK]] 2023-09-04 07:59:34,436 DEBUG [Listener at localhost/44221] wal.AbstractFSWAL(716): hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814364330 is not closed yet, will try archiving it next time 2023-09-04 07:59:34,436 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-04 07:59:34,436 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/WALs/jenkins-hbase4.apache.org,35915,1693814312663/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814354172 to hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/oldWALs/jenkins-hbase4.apache.org%2C35915%2C1693814312663.1693814354172 2023-09-04 07:59:34,436 INFO [Listener at localhost/44221] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-04 07:59:34,437 DEBUG [Listener at localhost/44221] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2f4bd615 to 127.0.0.1:49378 2023-09-04 07:59:34,437 DEBUG [Listener at localhost/44221] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:59:34,437 DEBUG [Listener at localhost/44221] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-04 07:59:34,437 DEBUG [Listener at localhost/44221] util.JVMClusterUtil(257): Found active master hash=1056118576, stopped=false 2023-09-04 07:59:34,437 INFO [Listener at localhost/44221] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:59:34,441 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 07:59:34,441 INFO [Listener at localhost/44221] procedure2.ProcedureExecutor(628): Stopping 2023-09-04 07:59:34,441 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:34,441 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 07:59:34,441 DEBUG [Listener at localhost/44221] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4f27143d to 127.0.0.1:49378 2023-09-04 07:59:34,442 DEBUG [Listener at localhost/44221] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:59:34,442 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:59:34,442 INFO [Listener at localhost/44221] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,35915,1693814312663' ***** 2023-09-04 07:59:34,442 INFO [Listener at localhost/44221] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-04 07:59:34,442 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:59:34,442 INFO [RS:0;jenkins-hbase4:35915] regionserver.HeapMemoryManager(220): Stopping 2023-09-04 07:59:34,443 INFO [RS:0;jenkins-hbase4:35915] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-04 07:59:34,443 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-04 07:59:34,443 INFO [RS:0;jenkins-hbase4:35915] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-04 07:59:34,443 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(3305): Received CLOSE for e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:59:34,443 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(3305): Received CLOSE for 07ec41a55ef66ec12093c83fd250c291 2023-09-04 07:59:34,443 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:34,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e85e122dc7ac3f058aa24020708ecde9, disabling compactions & flushes 2023-09-04 07:59:34,443 DEBUG [RS:0;jenkins-hbase4:35915] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x15956188 to 127.0.0.1:49378 2023-09-04 07:59:34,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:34,444 DEBUG [RS:0;jenkins-hbase4:35915] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:59:34,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:34,444 INFO [RS:0;jenkins-hbase4:35915] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-04 07:59:34,444 INFO [RS:0;jenkins-hbase4:35915] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-04 07:59:34,444 INFO [RS:0;jenkins-hbase4:35915] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-04 07:59:34,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. after waiting 0 ms 2023-09-04 07:59:34,444 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-04 07:59:34,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:34,444 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1474): Waiting on 3 regions to close 2023-09-04 07:59:34,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e85e122dc7ac3f058aa24020708ecde9 1/1 column families, dataSize=1.03 KB heapSize=1.35 KB 2023-09-04 07:59:34,444 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1478): Online Regions={e85e122dc7ac3f058aa24020708ecde9=TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9., 1588230740=hbase:meta,,1.1588230740, 07ec41a55ef66ec12093c83fd250c291=hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291.} 2023-09-04 07:59:34,444 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 07:59:34,444 DEBUG [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1504): Waiting on 07ec41a55ef66ec12093c83fd250c291, 1588230740, e85e122dc7ac3f058aa24020708ecde9 2023-09-04 07:59:34,444 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 07:59:34,445 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 07:59:34,445 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 07:59:34,445 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 07:59:34,445 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.10 KB heapSize=5.61 KB 2023-09-04 07:59:34,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.03 KB at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/f2d75140c49b45b1935fad28e54206d2 2023-09-04 07:59:34,463 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.84 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/.tmp/info/a99c87c6497a49d98f7854e6f066f1c3 2023-09-04 07:59:34,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/.tmp/info/f2d75140c49b45b1935fad28e54206d2 as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/f2d75140c49b45b1935fad28e54206d2 2023-09-04 07:59:34,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/f2d75140c49b45b1935fad28e54206d2, entries=1, sequenceid=22, filesize=5.8 K 2023-09-04 07:59:34,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.03 KB/1051, heapSize ~1.34 KB/1368, currentSize=0 B/0 for e85e122dc7ac3f058aa24020708ecde9 in 49ms, sequenceid=22, compaction requested=true 2023-09-04 07:59:34,497 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/1d981496b1c24544af4ef3bd0cc6a8b9, hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/9ad7b684b27d4472bf7cef4697d427ce, hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/3fcd969e805544a3a04b220bbb7024e8] to archive 2023-09-04 07:59:34,499 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 07:59:34,499 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=264 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/.tmp/table/800e355c070d4df4a7d6a38f1c77f774 2023-09-04 07:59:34,503 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/1d981496b1c24544af4ef3bd0cc6a8b9 to hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/1d981496b1c24544af4ef3bd0cc6a8b9 2023-09-04 07:59:34,505 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/9ad7b684b27d4472bf7cef4697d427ce to hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/9ad7b684b27d4472bf7cef4697d427ce 2023-09-04 07:59:34,508 DEBUG [StoreCloser-TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/3fcd969e805544a3a04b220bbb7024e8 to hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/archive/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/info/3fcd969e805544a3a04b220bbb7024e8 2023-09-04 07:59:34,508 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/.tmp/info/a99c87c6497a49d98f7854e6f066f1c3 as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/info/a99c87c6497a49d98f7854e6f066f1c3 2023-09-04 07:59:34,516 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/info/a99c87c6497a49d98f7854e6f066f1c3, entries=20, sequenceid=14, filesize=7.6 K 2023-09-04 07:59:34,518 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/.tmp/table/800e355c070d4df4a7d6a38f1c77f774 as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/table/800e355c070d4df4a7d6a38f1c77f774 2023-09-04 07:59:34,525 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/table/800e355c070d4df4a7d6a38f1c77f774, entries=4, sequenceid=14, filesize=4.9 K 2023-09-04 07:59:34,526 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.10 KB/3174, heapSize ~5.33 KB/5456, currentSize=0 B/0 for 1588230740 in 81ms, sequenceid=14, compaction requested=false 2023-09-04 07:59:34,534 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-09-04 07:59:34,535 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 07:59:34,538 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 07:59:34,538 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 07:59:34,538 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-04 07:59:34,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/default/TestLogRolling-testCompactionRecordDoesntBlockRolling/e85e122dc7ac3f058aa24020708ecde9/recovered.edits/25.seqid, newMaxSeqId=25, maxSeqId=1 2023-09-04 07:59:34,542 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:34,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e85e122dc7ac3f058aa24020708ecde9: 2023-09-04 07:59:34,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testCompactionRecordDoesntBlockRolling,,1693814313916.e85e122dc7ac3f058aa24020708ecde9. 2023-09-04 07:59:34,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 07ec41a55ef66ec12093c83fd250c291, disabling compactions & flushes 2023-09-04 07:59:34,543 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:59:34,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:59:34,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. after waiting 0 ms 2023-09-04 07:59:34,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:59:34,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/data/hbase/namespace/07ec41a55ef66ec12093c83fd250c291/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 07:59:34,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:59:34,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 07ec41a55ef66ec12093c83fd250c291: 2023-09-04 07:59:34,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693814313365.07ec41a55ef66ec12093c83fd250c291. 2023-09-04 07:59:34,645 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,35915,1693814312663; all regions closed. 2023-09-04 07:59:34,650 DEBUG [RS:0;jenkins-hbase4:35915] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/oldWALs 2023-09-04 07:59:34,650 INFO [RS:0;jenkins-hbase4:35915] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35915%2C1693814312663.meta:.meta(num 1693814313295) 2023-09-04 07:59:34,656 DEBUG [RS:0;jenkins-hbase4:35915] wal.AbstractFSWAL(1028): Moved 2 WAL file(s) to /user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/oldWALs 2023-09-04 07:59:34,656 INFO [RS:0;jenkins-hbase4:35915] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35915%2C1693814312663:(num 1693814374419) 2023-09-04 07:59:34,656 DEBUG [RS:0;jenkins-hbase4:35915] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:59:34,656 INFO [RS:0;jenkins-hbase4:35915] regionserver.LeaseManager(133): Closed leases 2023-09-04 07:59:34,656 INFO [RS:0;jenkins-hbase4:35915] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-09-04 07:59:34,657 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 07:59:34,657 INFO [RS:0;jenkins-hbase4:35915] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35915 2023-09-04 07:59:34,660 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,35915,1693814312663 2023-09-04 07:59:34,660 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 07:59:34,660 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 07:59:34,663 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35915,1693814312663] 2023-09-04 07:59:34,663 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35915,1693814312663; numProcessing=1 2023-09-04 07:59:34,664 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,35915,1693814312663 already deleted, retry=false 2023-09-04 07:59:34,664 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35915,1693814312663 expired; onlineServers=0 2023-09-04 07:59:34,664 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33811,1693814312612' ***** 2023-09-04 07:59:34,664 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-04 07:59:34,664 DEBUG [M:0;jenkins-hbase4:33811] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3acc8436, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 07:59:34,665 INFO [M:0;jenkins-hbase4:33811] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33811,1693814312612 2023-09-04 07:59:34,665 INFO [M:0;jenkins-hbase4:33811] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33811,1693814312612; all regions closed. 2023-09-04 07:59:34,665 DEBUG [M:0;jenkins-hbase4:33811] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 07:59:34,665 DEBUG [M:0;jenkins-hbase4:33811] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-04 07:59:34,665 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-04 07:59:34,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814312861] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814312861,5,FailOnTimeoutGroup] 2023-09-04 07:59:34,665 DEBUG [M:0;jenkins-hbase4:33811] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-04 07:59:34,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814312861] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814312861,5,FailOnTimeoutGroup] 2023-09-04 07:59:34,665 INFO [M:0;jenkins-hbase4:33811] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-04 07:59:34,666 INFO [M:0;jenkins-hbase4:33811] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-04 07:59:34,666 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-04 07:59:34,666 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:34,666 INFO [M:0;jenkins-hbase4:33811] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-04 07:59:34,666 DEBUG [M:0;jenkins-hbase4:33811] master.HMaster(1512): Stopping service threads 2023-09-04 07:59:34,666 INFO [M:0;jenkins-hbase4:33811] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-04 07:59:34,666 ERROR [M:0;jenkins-hbase4:33811] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-09-04 07:59:34,666 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 07:59:34,667 INFO [M:0;jenkins-hbase4:33811] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-04 07:59:34,667 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-04 07:59:34,667 DEBUG [M:0;jenkins-hbase4:33811] zookeeper.ZKUtil(398): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-04 07:59:34,667 WARN [M:0;jenkins-hbase4:33811] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-04 07:59:34,667 INFO [M:0;jenkins-hbase4:33811] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-04 07:59:34,667 INFO [M:0;jenkins-hbase4:33811] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-04 07:59:34,668 DEBUG [M:0;jenkins-hbase4:33811] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 07:59:34,668 INFO [M:0;jenkins-hbase4:33811] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:59:34,668 DEBUG [M:0;jenkins-hbase4:33811] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:59:34,668 DEBUG [M:0;jenkins-hbase4:33811] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 07:59:34,668 DEBUG [M:0;jenkins-hbase4:33811] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:59:34,668 INFO [M:0;jenkins-hbase4:33811] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=38.89 KB heapSize=47.33 KB 2023-09-04 07:59:34,680 INFO [M:0;jenkins-hbase4:33811] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=38.89 KB at sequenceid=100 (bloomFilter=true), to=hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/9277d40f70bf42a181db3d6f7ab5fc17 2023-09-04 07:59:34,686 INFO [M:0;jenkins-hbase4:33811] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9277d40f70bf42a181db3d6f7ab5fc17 2023-09-04 07:59:34,687 DEBUG [M:0;jenkins-hbase4:33811] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/9277d40f70bf42a181db3d6f7ab5fc17 as hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9277d40f70bf42a181db3d6f7ab5fc17 2023-09-04 07:59:34,692 INFO [M:0;jenkins-hbase4:33811] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9277d40f70bf42a181db3d6f7ab5fc17 2023-09-04 07:59:34,692 INFO [M:0;jenkins-hbase4:33811] regionserver.HStore(1080): Added hdfs://localhost:39673/user/jenkins/test-data/cc56db2a-01eb-d4d2-fad0-7c6dfc07e7cc/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9277d40f70bf42a181db3d6f7ab5fc17, entries=11, sequenceid=100, filesize=6.1 K 2023-09-04 07:59:34,693 INFO [M:0;jenkins-hbase4:33811] regionserver.HRegion(2948): Finished flush of dataSize ~38.89 KB/39824, heapSize ~47.31 KB/48448, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 25ms, sequenceid=100, compaction requested=false 2023-09-04 07:59:34,695 INFO [M:0;jenkins-hbase4:33811] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:59:34,695 DEBUG [M:0;jenkins-hbase4:33811] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 07:59:34,697 INFO [M:0;jenkins-hbase4:33811] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-04 07:59:34,698 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 07:59:34,698 INFO [M:0;jenkins-hbase4:33811] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33811 2023-09-04 07:59:34,700 DEBUG [M:0;jenkins-hbase4:33811] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33811,1693814312612 already deleted, retry=false 2023-09-04 07:59:34,763 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 07:59:34,763 INFO [RS:0;jenkins-hbase4:35915] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,35915,1693814312663; zookeeper connection closed. 2023-09-04 07:59:34,763 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): regionserver:35915-0x10075f2862e0001, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 07:59:34,764 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@28e58692] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@28e58692 2023-09-04 07:59:34,764 INFO [Listener at localhost/44221] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-04 07:59:34,863 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 07:59:34,863 INFO [M:0;jenkins-hbase4:33811] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33811,1693814312612; zookeeper connection closed. 2023-09-04 07:59:34,863 DEBUG [Listener at localhost/44221-EventThread] zookeeper.ZKWatcher(600): master:33811-0x10075f2862e0000, quorum=127.0.0.1:49378, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 07:59:34,864 WARN [Listener at localhost/44221] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 07:59:34,869 INFO [Listener at localhost/44221] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:59:34,944 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-04 07:59:34,973 WARN [BP-1322763515-172.31.14.131-1693814312018 heartbeating to localhost/127.0.0.1:39673] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 07:59:34,973 WARN [BP-1322763515-172.31.14.131-1693814312018 heartbeating to localhost/127.0.0.1:39673] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1322763515-172.31.14.131-1693814312018 (Datanode Uuid 9c03b603-7daa-4919-8f7d-d2206dfaadc3) service to localhost/127.0.0.1:39673 2023-09-04 07:59:34,974 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/cluster_d0389aae-903d-8a9a-817d-7948e87b3c60/dfs/data/data3/current/BP-1322763515-172.31.14.131-1693814312018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:59:34,974 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/cluster_d0389aae-903d-8a9a-817d-7948e87b3c60/dfs/data/data4/current/BP-1322763515-172.31.14.131-1693814312018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:59:34,975 WARN [Listener at localhost/44221] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 07:59:34,982 INFO [Listener at localhost/44221] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:59:35,086 WARN [BP-1322763515-172.31.14.131-1693814312018 heartbeating to localhost/127.0.0.1:39673] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 07:59:35,086 WARN [BP-1322763515-172.31.14.131-1693814312018 heartbeating to localhost/127.0.0.1:39673] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1322763515-172.31.14.131-1693814312018 (Datanode Uuid bb6f7770-d5af-4835-a80d-4e752f983720) service to localhost/127.0.0.1:39673 2023-09-04 07:59:35,087 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/cluster_d0389aae-903d-8a9a-817d-7948e87b3c60/dfs/data/data1/current/BP-1322763515-172.31.14.131-1693814312018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:59:35,087 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/cluster_d0389aae-903d-8a9a-817d-7948e87b3c60/dfs/data/data2/current/BP-1322763515-172.31.14.131-1693814312018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 07:59:35,098 INFO [Listener at localhost/44221] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 07:59:35,210 INFO [Listener at localhost/44221] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-04 07:59:35,226 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-04 07:59:35,235 INFO [Listener at localhost/44221] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testCompactionRecordDoesntBlockRolling Thread=63 (was 78), OpenFileDescriptor=442 (was 467), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=83 (was 135), ProcessCount=167 (was 167), AvailableMemoryMB=8447 (was 8848) 2023-09-04 07:59:35,242 INFO [Listener at localhost/44221] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=64, OpenFileDescriptor=442, MaxFileDescriptor=60000, SystemLoadAverage=83, ProcessCount=167, AvailableMemoryMB=8447 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/hadoop.log.dir so I do NOT create it in target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e174b325-73c6-e4ed-e883-f82efd6224e6/hadoop.tmp.dir so I do NOT create it in target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/cluster_de6439f2-5638-761f-c571-fce95f32cc03, deleteOnExit=true 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/test.cache.data in system properties and HBase conf 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/hadoop.tmp.dir in system properties and HBase conf 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/hadoop.log.dir in system properties and HBase conf 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-04 07:59:35,243 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-04 07:59:35,244 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-04 07:59:35,244 DEBUG [Listener at localhost/44221] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-04 07:59:35,244 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-04 07:59:35,244 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-04 07:59:35,244 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-04 07:59:35,244 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 07:59:35,244 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-04 07:59:35,244 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-04 07:59:35,245 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 07:59:35,245 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 07:59:35,245 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-04 07:59:35,245 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/nfs.dump.dir in system properties and HBase conf 2023-09-04 07:59:35,245 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/java.io.tmpdir in system properties and HBase conf 2023-09-04 07:59:35,245 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 07:59:35,245 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-04 07:59:35,245 INFO [Listener at localhost/44221] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-04 07:59:35,250 WARN [Listener at localhost/44221] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 07:59:35,250 WARN [Listener at localhost/44221] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 07:59:35,290 WARN [Listener at localhost/44221] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:59:35,292 INFO [Listener at localhost/44221] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:59:35,296 INFO [Listener at localhost/44221] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/java.io.tmpdir/Jetty_localhost_41611_hdfs____.mp8659/webapp 2023-09-04 07:59:35,387 INFO [Listener at localhost/44221] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41611 2023-09-04 07:59:35,391 WARN [Listener at localhost/44221] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 07:59:35,392 WARN [Listener at localhost/44221] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 07:59:35,433 WARN [Listener at localhost/40219] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:59:35,446 WARN [Listener at localhost/40219] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:59:35,448 WARN [Listener at localhost/40219] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:59:35,449 INFO [Listener at localhost/40219] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:59:35,453 INFO [Listener at localhost/40219] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/java.io.tmpdir/Jetty_localhost_35385_datanode____37w9f9/webapp 2023-09-04 07:59:35,544 INFO [Listener at localhost/40219] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35385 2023-09-04 07:59:35,550 WARN [Listener at localhost/37797] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:59:35,560 WARN [Listener at localhost/37797] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 07:59:35,562 WARN [Listener at localhost/37797] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 07:59:35,563 INFO [Listener at localhost/37797] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 07:59:35,568 INFO [Listener at localhost/37797] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/java.io.tmpdir/Jetty_localhost_42475_datanode____d6xtx7/webapp 2023-09-04 07:59:35,643 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb27d1027932a70fb: Processing first storage report for DS-6d705145-03fa-43a7-9ae4-ecae9c98215a from datanode c777e33f-9737-4700-8f48-dd1b2d25cda9 2023-09-04 07:59:35,643 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb27d1027932a70fb: from storage DS-6d705145-03fa-43a7-9ae4-ecae9c98215a node DatanodeRegistration(127.0.0.1:43733, datanodeUuid=c777e33f-9737-4700-8f48-dd1b2d25cda9, infoPort=41535, infoSecurePort=0, ipcPort=37797, storageInfo=lv=-57;cid=testClusterID;nsid=966310425;c=1693814375252), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-04 07:59:35,643 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb27d1027932a70fb: Processing first storage report for DS-cfaeb3ee-86ed-4c09-94de-873674deda85 from datanode c777e33f-9737-4700-8f48-dd1b2d25cda9 2023-09-04 07:59:35,643 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb27d1027932a70fb: from storage DS-cfaeb3ee-86ed-4c09-94de-873674deda85 node DatanodeRegistration(127.0.0.1:43733, datanodeUuid=c777e33f-9737-4700-8f48-dd1b2d25cda9, infoPort=41535, infoSecurePort=0, ipcPort=37797, storageInfo=lv=-57;cid=testClusterID;nsid=966310425;c=1693814375252), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:59:35,661 INFO [Listener at localhost/37797] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42475 2023-09-04 07:59:35,668 WARN [Listener at localhost/41609] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 07:59:35,765 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xaeff224788263d4f: Processing first storage report for DS-4cd26429-99e6-4591-ba18-6922364ff8e1 from datanode 9a39d184-686f-466b-8c3c-16e4016500f0 2023-09-04 07:59:35,765 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xaeff224788263d4f: from storage DS-4cd26429-99e6-4591-ba18-6922364ff8e1 node DatanodeRegistration(127.0.0.1:40945, datanodeUuid=9a39d184-686f-466b-8c3c-16e4016500f0, infoPort=37095, infoSecurePort=0, ipcPort=41609, storageInfo=lv=-57;cid=testClusterID;nsid=966310425;c=1693814375252), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-09-04 07:59:35,765 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xaeff224788263d4f: Processing first storage report for DS-66514c29-fb0c-4c04-ae82-cccd0b58133a from datanode 9a39d184-686f-466b-8c3c-16e4016500f0 2023-09-04 07:59:35,765 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xaeff224788263d4f: from storage DS-66514c29-fb0c-4c04-ae82-cccd0b58133a node DatanodeRegistration(127.0.0.1:40945, datanodeUuid=9a39d184-686f-466b-8c3c-16e4016500f0, infoPort=37095, infoSecurePort=0, ipcPort=41609, storageInfo=lv=-57;cid=testClusterID;nsid=966310425;c=1693814375252), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 07:59:35,777 DEBUG [Listener at localhost/41609] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232 2023-09-04 07:59:35,784 INFO [Listener at localhost/41609] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/cluster_de6439f2-5638-761f-c571-fce95f32cc03/zookeeper_0, clientPort=62260, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/cluster_de6439f2-5638-761f-c571-fce95f32cc03/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/cluster_de6439f2-5638-761f-c571-fce95f32cc03/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-04 07:59:35,785 INFO [Listener at localhost/41609] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=62260 2023-09-04 07:59:35,786 INFO [Listener at localhost/41609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:59:35,787 INFO [Listener at localhost/41609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:59:35,807 INFO [Listener at localhost/41609] util.FSUtils(471): Created version file at hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318 with version=8 2023-09-04 07:59:35,807 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/hbase-staging 2023-09-04 07:59:35,809 INFO [Listener at localhost/41609] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 07:59:35,809 INFO [Listener at localhost/41609] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:59:35,810 INFO [Listener at localhost/41609] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 07:59:35,810 INFO [Listener at localhost/41609] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 07:59:35,810 INFO [Listener at localhost/41609] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:59:35,810 INFO [Listener at localhost/41609] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 07:59:35,810 INFO [Listener at localhost/41609] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 07:59:35,812 INFO [Listener at localhost/41609] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45823 2023-09-04 07:59:35,812 INFO [Listener at localhost/41609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:59:35,813 INFO [Listener at localhost/41609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:59:35,814 INFO [Listener at localhost/41609] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45823 connecting to ZooKeeper ensemble=127.0.0.1:62260 2023-09-04 07:59:35,824 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:458230x0, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 07:59:35,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45823-0x10075f37d0b0000 connected 2023-09-04 07:59:35,843 DEBUG [Listener at localhost/41609] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 07:59:35,843 DEBUG [Listener at localhost/41609] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:59:35,844 DEBUG [Listener at localhost/41609] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 07:59:35,846 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45823 2023-09-04 07:59:35,848 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45823 2023-09-04 07:59:35,850 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45823 2023-09-04 07:59:35,857 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45823 2023-09-04 07:59:35,859 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45823 2023-09-04 07:59:35,859 INFO [Listener at localhost/41609] master.HMaster(444): hbase.rootdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318, hbase.cluster.distributed=false 2023-09-04 07:59:35,874 INFO [Listener at localhost/41609] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 07:59:35,874 INFO [Listener at localhost/41609] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:59:35,875 INFO [Listener at localhost/41609] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 07:59:35,875 INFO [Listener at localhost/41609] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 07:59:35,875 INFO [Listener at localhost/41609] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 07:59:35,875 INFO [Listener at localhost/41609] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 07:59:35,875 INFO [Listener at localhost/41609] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 07:59:35,876 INFO [Listener at localhost/41609] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40899 2023-09-04 07:59:35,877 INFO [Listener at localhost/41609] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-04 07:59:35,878 DEBUG [Listener at localhost/41609] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-04 07:59:35,878 INFO [Listener at localhost/41609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:59:35,879 INFO [Listener at localhost/41609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:59:35,880 INFO [Listener at localhost/41609] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40899 connecting to ZooKeeper ensemble=127.0.0.1:62260 2023-09-04 07:59:35,884 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): regionserver:408990x0, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 07:59:35,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40899-0x10075f37d0b0001 connected 2023-09-04 07:59:35,885 DEBUG [Listener at localhost/41609] zookeeper.ZKUtil(164): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 07:59:35,886 DEBUG [Listener at localhost/41609] zookeeper.ZKUtil(164): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 07:59:35,886 DEBUG [Listener at localhost/41609] zookeeper.ZKUtil(164): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 07:59:35,887 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40899 2023-09-04 07:59:35,887 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40899 2023-09-04 07:59:35,887 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40899 2023-09-04 07:59:35,887 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40899 2023-09-04 07:59:35,888 DEBUG [Listener at localhost/41609] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40899 2023-09-04 07:59:35,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 07:59:35,890 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 07:59:35,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 07:59:35,891 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 07:59:35,891 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 07:59:35,891 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:35,892 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 07:59:35,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 07:59:35,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,45823,1693814375809 from backup master directory 2023-09-04 07:59:35,894 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 07:59:35,894 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 07:59:35,894 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 07:59:35,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 07:59:35,908 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/hbase.id with ID: 228f2eba-7b28-4e0a-960a-a15a16ab25be 2023-09-04 07:59:35,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:59:35,924 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:35,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x59250f99 to 127.0.0.1:62260 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:59:35,936 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2aa0ef25, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:59:35,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 07:59:35,937 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-04 07:59:35,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:59:35,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store-tmp 2023-09-04 07:59:35,957 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:59:35,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 07:59:35,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:59:35,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:59:35,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 07:59:35,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:59:35,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 07:59:35,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 07:59:35,959 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/WALs/jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 07:59:35,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45823%2C1693814375809, suffix=, logDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/WALs/jenkins-hbase4.apache.org,45823,1693814375809, archiveDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/oldWALs, maxLogs=10 2023-09-04 07:59:35,983 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK] 2023-09-04 07:59:35,983 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK] 2023-09-04 07:59:35,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/WALs/jenkins-hbase4.apache.org,45823,1693814375809/jenkins-hbase4.apache.org%2C45823%2C1693814375809.1693814375962 2023-09-04 07:59:35,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK], DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK]] 2023-09-04 07:59:35,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:59:35,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:59:35,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:59:35,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:59:35,988 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:59:35,990 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-04 07:59:35,990 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-04 07:59:35,991 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:35,992 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:59:35,993 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:59:35,995 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 07:59:35,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:59:35,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=735946, jitterRate=-0.06419719755649567}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:59:35,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 07:59:35,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-04 07:59:35,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-04 07:59:36,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-04 07:59:36,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-04 07:59:36,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-04 07:59:36,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-04 07:59:36,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-04 07:59:36,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-04 07:59:36,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-04 07:59:36,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-04 07:59:36,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-04 07:59:36,014 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-04 07:59:36,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-04 07:59:36,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-04 07:59:36,016 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:36,017 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-04 07:59:36,018 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-04 07:59:36,019 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-04 07:59:36,020 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 07:59:36,020 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 07:59:36,020 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:36,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,45823,1693814375809, sessionid=0x10075f37d0b0000, setting cluster-up flag (Was=false) 2023-09-04 07:59:36,026 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:36,031 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-04 07:59:36,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 07:59:36,036 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:36,040 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-04 07:59:36,041 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 07:59:36,042 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.hbase-snapshot/.tmp 2023-09-04 07:59:36,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-04 07:59:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:59:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:59:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:59:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 07:59:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-04 07:59:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 07:59:36,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693814406053 2023-09-04 07:59:36,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-04 07:59:36,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-04 07:59:36,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-04 07:59:36,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-04 07:59:36,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-04 07:59:36,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-04 07:59:36,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,061 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 07:59:36,062 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-04 07:59:36,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-04 07:59:36,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-04 07:59:36,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-04 07:59:36,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-04 07:59:36,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-04 07:59:36,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814376062,5,FailOnTimeoutGroup] 2023-09-04 07:59:36,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814376062,5,FailOnTimeoutGroup] 2023-09-04 07:59:36,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-04 07:59:36,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,063 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 07:59:36,077 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 07:59:36,078 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 07:59:36,078 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318 2023-09-04 07:59:36,090 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:59:36,090 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(951): ClusterId : 228f2eba-7b28-4e0a-960a-a15a16ab25be 2023-09-04 07:59:36,091 DEBUG [RS:0;jenkins-hbase4:40899] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-04 07:59:36,092 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 07:59:36,093 DEBUG [RS:0;jenkins-hbase4:40899] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-04 07:59:36,093 DEBUG [RS:0;jenkins-hbase4:40899] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-04 07:59:36,093 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/info 2023-09-04 07:59:36,094 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 07:59:36,094 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:36,095 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 07:59:36,095 DEBUG [RS:0;jenkins-hbase4:40899] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-04 07:59:36,096 DEBUG [RS:0;jenkins-hbase4:40899] zookeeper.ReadOnlyZKClient(139): Connect 0x2cc9af44 to 127.0.0.1:62260 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:59:36,097 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:59:36,098 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 07:59:36,099 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:36,099 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 07:59:36,100 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/table 2023-09-04 07:59:36,100 DEBUG [RS:0;jenkins-hbase4:40899] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@220e54ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:59:36,101 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 07:59:36,101 DEBUG [RS:0;jenkins-hbase4:40899] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2631c434, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 07:59:36,102 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:36,103 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740 2023-09-04 07:59:36,103 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740 2023-09-04 07:59:36,106 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 07:59:36,107 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 07:59:36,110 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:59:36,111 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=700021, jitterRate=-0.10987760126590729}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 07:59:36,111 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 07:59:36,111 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 07:59:36,111 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 07:59:36,111 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 07:59:36,111 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 07:59:36,111 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 07:59:36,112 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 07:59:36,112 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 07:59:36,112 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:40899 2023-09-04 07:59:36,112 INFO [RS:0;jenkins-hbase4:40899] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-04 07:59:36,112 INFO [RS:0;jenkins-hbase4:40899] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-04 07:59:36,112 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1022): About to register with Master. 2023-09-04 07:59:36,113 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45823,1693814375809 with isa=jenkins-hbase4.apache.org/172.31.14.131:40899, startcode=1693814375874 2023-09-04 07:59:36,113 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 07:59:36,113 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-04 07:59:36,113 DEBUG [RS:0;jenkins-hbase4:40899] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-04 07:59:36,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-04 07:59:36,115 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-04 07:59:36,116 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48275, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-09-04 07:59:36,116 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-04 07:59:36,117 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45823] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,118 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318 2023-09-04 07:59:36,118 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:40219 2023-09-04 07:59:36,118 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-04 07:59:36,119 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 07:59:36,120 DEBUG [RS:0;jenkins-hbase4:40899] zookeeper.ZKUtil(162): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,120 WARN [RS:0;jenkins-hbase4:40899] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 07:59:36,120 INFO [RS:0;jenkins-hbase4:40899] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:59:36,120 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1948): logDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,126 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40899,1693814375874] 2023-09-04 07:59:36,130 DEBUG [RS:0;jenkins-hbase4:40899] zookeeper.ZKUtil(162): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,131 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-04 07:59:36,131 INFO [RS:0;jenkins-hbase4:40899] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-04 07:59:36,133 INFO [RS:0;jenkins-hbase4:40899] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-04 07:59:36,136 INFO [RS:0;jenkins-hbase4:40899] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 07:59:36,136 INFO [RS:0;jenkins-hbase4:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,136 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-04 07:59:36,138 INFO [RS:0;jenkins-hbase4:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,138 DEBUG [RS:0;jenkins-hbase4:40899] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 07:59:36,141 INFO [RS:0;jenkins-hbase4:40899] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,141 INFO [RS:0;jenkins-hbase4:40899] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,141 INFO [RS:0;jenkins-hbase4:40899] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,152 INFO [RS:0;jenkins-hbase4:40899] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-04 07:59:36,153 INFO [RS:0;jenkins-hbase4:40899] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40899,1693814375874-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,164 INFO [RS:0;jenkins-hbase4:40899] regionserver.Replication(203): jenkins-hbase4.apache.org,40899,1693814375874 started 2023-09-04 07:59:36,164 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,40899,1693814375874, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40899, sessionid=0x10075f37d0b0001 2023-09-04 07:59:36,164 DEBUG [RS:0;jenkins-hbase4:40899] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-04 07:59:36,164 DEBUG [RS:0;jenkins-hbase4:40899] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,164 DEBUG [RS:0;jenkins-hbase4:40899] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40899,1693814375874' 2023-09-04 07:59:36,164 DEBUG [RS:0;jenkins-hbase4:40899] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 07:59:36,164 DEBUG [RS:0;jenkins-hbase4:40899] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 07:59:36,165 DEBUG [RS:0;jenkins-hbase4:40899] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-04 07:59:36,165 DEBUG [RS:0;jenkins-hbase4:40899] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-04 07:59:36,165 DEBUG [RS:0;jenkins-hbase4:40899] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,165 DEBUG [RS:0;jenkins-hbase4:40899] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40899,1693814375874' 2023-09-04 07:59:36,165 DEBUG [RS:0;jenkins-hbase4:40899] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-04 07:59:36,165 DEBUG [RS:0;jenkins-hbase4:40899] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-04 07:59:36,166 DEBUG [RS:0;jenkins-hbase4:40899] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-04 07:59:36,166 INFO [RS:0;jenkins-hbase4:40899] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-04 07:59:36,166 INFO [RS:0;jenkins-hbase4:40899] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-04 07:59:36,267 DEBUG [jenkins-hbase4:45823] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-04 07:59:36,268 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40899,1693814375874, state=OPENING 2023-09-04 07:59:36,269 INFO [RS:0;jenkins-hbase4:40899] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40899%2C1693814375874, suffix=, logDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874, archiveDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/oldWALs, maxLogs=32 2023-09-04 07:59:36,270 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-04 07:59:36,271 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:36,271 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 07:59:36,271 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40899,1693814375874}] 2023-09-04 07:59:36,288 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK] 2023-09-04 07:59:36,288 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK] 2023-09-04 07:59:36,290 INFO [RS:0;jenkins-hbase4:40899] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814376270 2023-09-04 07:59:36,290 DEBUG [RS:0;jenkins-hbase4:40899] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK], DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK]] 2023-09-04 07:59:36,426 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,426 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-04 07:59:36,428 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39442, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-04 07:59:36,432 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-04 07:59:36,432 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 07:59:36,434 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40899%2C1693814375874.meta, suffix=.meta, logDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874, archiveDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/oldWALs, maxLogs=32 2023-09-04 07:59:36,449 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK] 2023-09-04 07:59:36,449 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK] 2023-09-04 07:59:36,451 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.meta.1693814376435.meta 2023-09-04 07:59:36,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK], DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK]] 2023-09-04 07:59:36,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:59:36,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-04 07:59:36,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-04 07:59:36,452 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-04 07:59:36,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-04 07:59:36,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:59:36,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-04 07:59:36,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-04 07:59:36,453 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 07:59:36,455 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/info 2023-09-04 07:59:36,455 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/info 2023-09-04 07:59:36,455 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 07:59:36,456 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:36,456 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 07:59:36,457 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:59:36,457 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/rep_barrier 2023-09-04 07:59:36,457 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 07:59:36,458 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:36,458 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 07:59:36,459 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/table 2023-09-04 07:59:36,459 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/table 2023-09-04 07:59:36,459 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 07:59:36,460 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:36,461 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740 2023-09-04 07:59:36,463 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740 2023-09-04 07:59:36,465 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 07:59:36,467 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 07:59:36,468 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=803414, jitterRate=0.02159479260444641}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 07:59:36,468 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 07:59:36,469 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693814376426 2023-09-04 07:59:36,474 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-04 07:59:36,474 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-04 07:59:36,475 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40899,1693814375874, state=OPEN 2023-09-04 07:59:36,477 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-04 07:59:36,478 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 07:59:36,480 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-04 07:59:36,480 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40899,1693814375874 in 207 msec 2023-09-04 07:59:36,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-04 07:59:36,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 367 msec 2023-09-04 07:59:36,484 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 440 msec 2023-09-04 07:59:36,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693814376484, completionTime=-1 2023-09-04 07:59:36,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-04 07:59:36,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-04 07:59:36,488 DEBUG [hconnection-0x7a2f9b41-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 07:59:36,489 INFO [RS-EventLoopGroup-8-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39456, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 07:59:36,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-04 07:59:36,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693814436491 2023-09-04 07:59:36,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693814496491 2023-09-04 07:59:36,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-09-04 07:59:36,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45823,1693814375809-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45823,1693814375809-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45823,1693814375809-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45823, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-04 07:59:36,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-04 07:59:36,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 07:59:36,502 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-04 07:59:36,502 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-04 07:59:36,504 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 07:59:36,505 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 07:59:36,507 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,508 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf empty. 2023-09-04 07:59:36,508 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,508 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-04 07:59:36,520 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-04 07:59:36,522 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3744a879ceb8a9f6ad86aeb3b6fb4bbf, NAME => 'hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp 2023-09-04 07:59:36,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:59:36,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 3744a879ceb8a9f6ad86aeb3b6fb4bbf, disabling compactions & flushes 2023-09-04 07:59:36,531 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 07:59:36,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 07:59:36,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. after waiting 0 ms 2023-09-04 07:59:36,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 07:59:36,531 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 07:59:36,531 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 3744a879ceb8a9f6ad86aeb3b6fb4bbf: 2023-09-04 07:59:36,533 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 07:59:36,535 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814376535"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814376535"}]},"ts":"1693814376535"} 2023-09-04 07:59:36,537 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 07:59:36,538 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 07:59:36,538 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814376538"}]},"ts":"1693814376538"} 2023-09-04 07:59:36,540 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-04 07:59:36,548 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3744a879ceb8a9f6ad86aeb3b6fb4bbf, ASSIGN}] 2023-09-04 07:59:36,550 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3744a879ceb8a9f6ad86aeb3b6fb4bbf, ASSIGN 2023-09-04 07:59:36,551 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=3744a879ceb8a9f6ad86aeb3b6fb4bbf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40899,1693814375874; forceNewPlan=false, retain=false 2023-09-04 07:59:36,702 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=3744a879ceb8a9f6ad86aeb3b6fb4bbf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,702 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814376702"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814376702"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814376702"}]},"ts":"1693814376702"} 2023-09-04 07:59:36,704 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 3744a879ceb8a9f6ad86aeb3b6fb4bbf, server=jenkins-hbase4.apache.org,40899,1693814375874}] 2023-09-04 07:59:36,860 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 07:59:36,861 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3744a879ceb8a9f6ad86aeb3b6fb4bbf, NAME => 'hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:59:36,861 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,861 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:59:36,861 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,861 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,863 INFO [StoreOpener-3744a879ceb8a9f6ad86aeb3b6fb4bbf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,864 DEBUG [StoreOpener-3744a879ceb8a9f6ad86aeb3b6fb4bbf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf/info 2023-09-04 07:59:36,864 DEBUG [StoreOpener-3744a879ceb8a9f6ad86aeb3b6fb4bbf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf/info 2023-09-04 07:59:36,865 INFO [StoreOpener-3744a879ceb8a9f6ad86aeb3b6fb4bbf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3744a879ceb8a9f6ad86aeb3b6fb4bbf columnFamilyName info 2023-09-04 07:59:36,865 INFO [StoreOpener-3744a879ceb8a9f6ad86aeb3b6fb4bbf-1] regionserver.HStore(310): Store=3744a879ceb8a9f6ad86aeb3b6fb4bbf/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:36,866 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,867 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,871 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 07:59:36,873 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:59:36,874 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3744a879ceb8a9f6ad86aeb3b6fb4bbf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=849751, jitterRate=0.08051469922065735}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:59:36,874 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3744a879ceb8a9f6ad86aeb3b6fb4bbf: 2023-09-04 07:59:36,875 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf., pid=6, masterSystemTime=1693814376857 2023-09-04 07:59:36,877 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 07:59:36,877 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 07:59:36,878 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=3744a879ceb8a9f6ad86aeb3b6fb4bbf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:36,878 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814376878"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814376878"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814376878"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814376878"}]},"ts":"1693814376878"} 2023-09-04 07:59:36,882 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-04 07:59:36,882 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 3744a879ceb8a9f6ad86aeb3b6fb4bbf, server=jenkins-hbase4.apache.org,40899,1693814375874 in 176 msec 2023-09-04 07:59:36,884 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-04 07:59:36,885 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=3744a879ceb8a9f6ad86aeb3b6fb4bbf, ASSIGN in 334 msec 2023-09-04 07:59:36,886 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 07:59:36,887 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814376887"}]},"ts":"1693814376887"} 2023-09-04 07:59:36,888 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-04 07:59:36,890 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 07:59:36,892 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 390 msec 2023-09-04 07:59:36,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-04 07:59:36,904 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:59:36,904 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:36,908 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-04 07:59:36,917 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:59:36,920 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 12 msec 2023-09-04 07:59:36,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-04 07:59:36,938 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 07:59:36,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 11 msec 2023-09-04 07:59:36,956 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-04 07:59:36,958 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-04 07:59:36,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 1.064sec 2023-09-04 07:59:36,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-04 07:59:36,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-04 07:59:36,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-04 07:59:36,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45823,1693814375809-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-04 07:59:36,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45823,1693814375809-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-04 07:59:36,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-04 07:59:36,990 DEBUG [Listener at localhost/41609] zookeeper.ReadOnlyZKClient(139): Connect 0x369b80bf to 127.0.0.1:62260 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 07:59:36,995 DEBUG [Listener at localhost/41609] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@620898ee, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 07:59:36,997 DEBUG [hconnection-0x12d28a86-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 07:59:36,999 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39458, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 07:59:37,000 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 07:59:37,001 INFO [Listener at localhost/41609] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 07:59:37,004 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-04 07:59:37,004 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 07:59:37,005 INFO [Listener at localhost/41609] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-04 07:59:37,007 DEBUG [Listener at localhost/41609] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-09-04 07:59:37,010 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42530, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-09-04 07:59:37,011 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45823] util.TableDescriptorChecker(340): MAX_FILESIZE for table descriptor or "hbase.hregion.max.filesize" (786432) is too small, which might cause over splitting into unmanageable number of regions. 2023-09-04 07:59:37,011 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45823] util.TableDescriptorChecker(340): MEMSTORE_FLUSHSIZE for table descriptor or "hbase.hregion.memstore.flush.size" (8192) is too small, which might cause very frequent flushing. 2023-09-04 07:59:37,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45823] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 07:59:37,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45823] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestLogRolling-testLogRolling 2023-09-04 07:59:37,015 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 07:59:37,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45823] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestLogRolling-testLogRolling" procId is: 9 2023-09-04 07:59:37,016 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 07:59:37,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45823] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 07:59:37,018 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,019 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0 empty. 2023-09-04 07:59:37,019 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,019 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestLogRolling-testLogRolling regions 2023-09-04 07:59:37,031 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp/data/default/TestLogRolling-testLogRolling/.tabledesc/.tableinfo.0000000001 2023-09-04 07:59:37,032 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(7675): creating {ENCODED => 29b143ee703034c3b91f7c15cae9dae0, NAME => 'TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestLogRolling-testLogRolling', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/.tmp 2023-09-04 07:59:37,042 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:59:37,042 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1604): Closing 29b143ee703034c3b91f7c15cae9dae0, disabling compactions & flushes 2023-09-04 07:59:37,042 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:37,042 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:37,042 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. after waiting 0 ms 2023-09-04 07:59:37,042 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:37,043 INFO [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:37,043 DEBUG [RegionOpenAndInit-TestLogRolling-testLogRolling-pool-0] regionserver.HRegion(1558): Region close journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 07:59:37,045 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 07:59:37,046 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693814377046"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814377046"}]},"ts":"1693814377046"} 2023-09-04 07:59:37,048 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 07:59:37,049 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 07:59:37,049 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814377049"}]},"ts":"1693814377049"} 2023-09-04 07:59:37,051 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLING in hbase:meta 2023-09-04 07:59:37,054 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=29b143ee703034c3b91f7c15cae9dae0, ASSIGN}] 2023-09-04 07:59:37,055 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=29b143ee703034c3b91f7c15cae9dae0, ASSIGN 2023-09-04 07:59:37,056 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=29b143ee703034c3b91f7c15cae9dae0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40899,1693814375874; forceNewPlan=false, retain=false 2023-09-04 07:59:37,207 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=29b143ee703034c3b91f7c15cae9dae0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:37,208 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693814377207"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814377207"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814377207"}]},"ts":"1693814377207"} 2023-09-04 07:59:37,210 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 29b143ee703034c3b91f7c15cae9dae0, server=jenkins-hbase4.apache.org,40899,1693814375874}] 2023-09-04 07:59:37,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:37,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 29b143ee703034c3b91f7c15cae9dae0, NAME => 'TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.', STARTKEY => '', ENDKEY => ''} 2023-09-04 07:59:37,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 07:59:37,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,368 INFO [StoreOpener-29b143ee703034c3b91f7c15cae9dae0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,370 DEBUG [StoreOpener-29b143ee703034c3b91f7c15cae9dae0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info 2023-09-04 07:59:37,370 DEBUG [StoreOpener-29b143ee703034c3b91f7c15cae9dae0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info 2023-09-04 07:59:37,370 INFO [StoreOpener-29b143ee703034c3b91f7c15cae9dae0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29b143ee703034c3b91f7c15cae9dae0 columnFamilyName info 2023-09-04 07:59:37,370 INFO [StoreOpener-29b143ee703034c3b91f7c15cae9dae0-1] regionserver.HStore(310): Store=29b143ee703034c3b91f7c15cae9dae0/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 07:59:37,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:37,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 07:59:37,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 29b143ee703034c3b91f7c15cae9dae0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=800007, jitterRate=0.017262622714042664}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 07:59:37,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 07:59:37,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0., pid=11, masterSystemTime=1693814377362 2023-09-04 07:59:37,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:37,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:37,382 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=29b143ee703034c3b91f7c15cae9dae0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:37,383 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693814377382"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814377382"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814377382"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814377382"}]},"ts":"1693814377382"} 2023-09-04 07:59:37,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-09-04 07:59:37,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 29b143ee703034c3b91f7c15cae9dae0, server=jenkins-hbase4.apache.org,40899,1693814375874 in 174 msec 2023-09-04 07:59:37,389 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-09-04 07:59:37,390 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=29b143ee703034c3b91f7c15cae9dae0, ASSIGN in 333 msec 2023-09-04 07:59:37,390 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 07:59:37,391 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestLogRolling-testLogRolling","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814377390"}]},"ts":"1693814377390"} 2023-09-04 07:59:37,392 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestLogRolling-testLogRolling, state=ENABLED in hbase:meta 2023-09-04 07:59:37,395 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestLogRolling-testLogRolling execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 07:59:37,397 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=TestLogRolling-testLogRolling in 383 msec 2023-09-04 07:59:40,067 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 07:59:42,131 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-09-04 07:59:42,131 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-09-04 07:59:42,132 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestLogRolling-testLogRolling' 2023-09-04 07:59:47,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45823] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-09-04 07:59:47,018 INFO [Listener at localhost/41609] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestLogRolling-testLogRolling, procId: 9 completed 2023-09-04 07:59:47,021 DEBUG [Listener at localhost/41609] hbase.HBaseTestingUtility(2627): Found 1 regions for table TestLogRolling-testLogRolling 2023-09-04 07:59:47,021 DEBUG [Listener at localhost/41609] hbase.HBaseTestingUtility(2633): firstRegionName=TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:47,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:47,038 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 29b143ee703034c3b91f7c15cae9dae0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 07:59:47,057 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=12 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/b687bd1d3a4c4b41a5f69e85bb9a8061 2023-09-04 07:59:47,073 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/b687bd1d3a4c4b41a5f69e85bb9a8061 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b687bd1d3a4c4b41a5f69e85bb9a8061 2023-09-04 07:59:47,077 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=29b143ee703034c3b91f7c15cae9dae0, server=jenkins-hbase4.apache.org,40899,1693814375874 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 07:59:47,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] ipc.CallRunner(144): callId: 38 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:39458 deadline: 1693814397077, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=29b143ee703034c3b91f7c15cae9dae0, server=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 07:59:47,080 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b687bd1d3a4c4b41a5f69e85bb9a8061, entries=8, sequenceid=12, filesize=13.0 K 2023-09-04 07:59:47,082 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 29b143ee703034c3b91f7c15cae9dae0 in 44ms, sequenceid=12, compaction requested=false 2023-09-04 07:59:47,082 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 07:59:57,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:57,103 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 29b143ee703034c3b91f7c15cae9dae0 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 07:59:57,119 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=37 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/bea6cf3a5a1547c98aa59d2f87a9d34d 2023-09-04 07:59:57,126 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/bea6cf3a5a1547c98aa59d2f87a9d34d as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bea6cf3a5a1547c98aa59d2f87a9d34d 2023-09-04 07:59:57,132 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bea6cf3a5a1547c98aa59d2f87a9d34d, entries=22, sequenceid=37, filesize=27.4 K 2023-09-04 07:59:57,133 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=2.05 KB/2102 for 29b143ee703034c3b91f7c15cae9dae0 in 30ms, sequenceid=37, compaction requested=false 2023-09-04 07:59:57,133 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 07:59:57,134 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=40.3 K, sizeToCheck=16.0 K 2023-09-04 07:59:57,134 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 07:59:57,135 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bea6cf3a5a1547c98aa59d2f87a9d34d because midkey is the same as first or last row 2023-09-04 07:59:59,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:59,116 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 29b143ee703034c3b91f7c15cae9dae0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 07:59:59,133 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=48 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/b686a481b9984fdf906843364a8f0267 2023-09-04 07:59:59,140 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/b686a481b9984fdf906843364a8f0267 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b686a481b9984fdf906843364a8f0267 2023-09-04 07:59:59,147 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b686a481b9984fdf906843364a8f0267, entries=8, sequenceid=48, filesize=13.0 K 2023-09-04 07:59:59,148 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=20.53 KB/21020 for 29b143ee703034c3b91f7c15cae9dae0 in 31ms, sequenceid=48, compaction requested=true 2023-09-04 07:59:59,148 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 07:59:59,148 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=53.3 K, sizeToCheck=16.0 K 2023-09-04 07:59:59,148 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 07:59:59,148 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bea6cf3a5a1547c98aa59d2f87a9d34d because midkey is the same as first or last row 2023-09-04 07:59:59,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 07:59:59,152 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 07:59:59,152 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 07:59:59,152 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 29b143ee703034c3b91f7c15cae9dae0 1/1 column families, dataSize=23.61 KB heapSize=25.59 KB 2023-09-04 07:59:59,154 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 54576 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 07:59:59,154 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): 29b143ee703034c3b91f7c15cae9dae0/info is initiating minor compaction (all files) 2023-09-04 07:59:59,154 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 29b143ee703034c3b91f7c15cae9dae0/info in TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 07:59:59,154 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b687bd1d3a4c4b41a5f69e85bb9a8061, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bea6cf3a5a1547c98aa59d2f87a9d34d, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b686a481b9984fdf906843364a8f0267] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp, totalSize=53.3 K 2023-09-04 07:59:59,155 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting b687bd1d3a4c4b41a5f69e85bb9a8061, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1693814387024 2023-09-04 07:59:59,156 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting bea6cf3a5a1547c98aa59d2f87a9d34d, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=37, earliestPutTs=1693814387039 2023-09-04 07:59:59,156 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting b686a481b9984fdf906843364a8f0267, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1693814397104 2023-09-04 07:59:59,167 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.63 KB at sequenceid=75 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/fa95dc614ed5462ba30d71bd5b7f902f 2023-09-04 07:59:59,173 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): 29b143ee703034c3b91f7c15cae9dae0#info#compaction#19 average throughput is 19.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 07:59:59,175 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/fa95dc614ed5462ba30d71bd5b7f902f as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/fa95dc614ed5462ba30d71bd5b7f902f 2023-09-04 07:59:59,192 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/fa95dc614ed5462ba30d71bd5b7f902f, entries=24, sequenceid=75, filesize=29.4 K 2023-09-04 07:59:59,193 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~24.63 KB/25224, heapSize ~26.67 KB/27312, currentSize=2.05 KB/2102 for 29b143ee703034c3b91f7c15cae9dae0 in 41ms, sequenceid=75, compaction requested=false 2023-09-04 07:59:59,193 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 07:59:59,194 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=82.7 K, sizeToCheck=16.0 K 2023-09-04 07:59:59,194 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 07:59:59,194 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/fa95dc614ed5462ba30d71bd5b7f902f because midkey is the same as first or last row 2023-09-04 07:59:59,197 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/bb6421bcffa24634abd5214adb56d85c as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bb6421bcffa24634abd5214adb56d85c 2023-09-04 07:59:59,204 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 29b143ee703034c3b91f7c15cae9dae0/info of 29b143ee703034c3b91f7c15cae9dae0 into bb6421bcffa24634abd5214adb56d85c(size=43.9 K), total size for store is 73.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 07:59:59,204 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 07:59:59,204 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0., storeName=29b143ee703034c3b91f7c15cae9dae0/info, priority=13, startTime=1693814399149; duration=0sec 2023-09-04 07:59:59,205 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=73.4 K, sizeToCheck=16.0 K 2023-09-04 07:59:59,205 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 07:59:59,205 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bb6421bcffa24634abd5214adb56d85c because midkey is the same as first or last row 2023-09-04 07:59:59,205 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:01,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:00:01,164 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 29b143ee703034c3b91f7c15cae9dae0 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 08:00:01,191 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=87 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/ff919acf66944b58b35cac5c46124a9c 2023-09-04 08:00:01,201 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/ff919acf66944b58b35cac5c46124a9c as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/ff919acf66944b58b35cac5c46124a9c 2023-09-04 08:00:01,206 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=29b143ee703034c3b91f7c15cae9dae0, server=jenkins-hbase4.apache.org,40899,1693814375874 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 08:00:01,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] ipc.CallRunner(144): callId: 102 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:39458 deadline: 1693814411206, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=29b143ee703034c3b91f7c15cae9dae0, server=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:01,208 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/ff919acf66944b58b35cac5c46124a9c, entries=8, sequenceid=87, filesize=13.0 K 2023-09-04 08:00:01,209 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for 29b143ee703034c3b91f7c15cae9dae0 in 44ms, sequenceid=87, compaction requested=true 2023-09-04 08:00:01,209 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 08:00:01,209 DEBUG [MemStoreFlusher.0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=86.3 K, sizeToCheck=16.0 K 2023-09-04 08:00:01,209 DEBUG [MemStoreFlusher.0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 08:00:01,209 DEBUG [MemStoreFlusher.0] regionserver.StoreUtils(129): cannot split hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bb6421bcffa24634abd5214adb56d85c because midkey is the same as first or last row 2023-09-04 08:00:01,209 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:01,209 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 08:00:01,211 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 88399 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 08:00:01,211 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): 29b143ee703034c3b91f7c15cae9dae0/info is initiating minor compaction (all files) 2023-09-04 08:00:01,211 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 29b143ee703034c3b91f7c15cae9dae0/info in TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 08:00:01,211 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bb6421bcffa24634abd5214adb56d85c, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/fa95dc614ed5462ba30d71bd5b7f902f, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/ff919acf66944b58b35cac5c46124a9c] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp, totalSize=86.3 K 2023-09-04 08:00:01,212 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting bb6421bcffa24634abd5214adb56d85c, keycount=38, bloomtype=ROW, size=43.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1693814387024 2023-09-04 08:00:01,212 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting fa95dc614ed5462ba30d71bd5b7f902f, keycount=24, bloomtype=ROW, size=29.4 K, encoding=NONE, compression=NONE, seqNum=75, earliestPutTs=1693814399116 2023-09-04 08:00:01,213 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting ff919acf66944b58b35cac5c46124a9c, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=87, earliestPutTs=1693814399153 2023-09-04 08:00:01,228 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): 29b143ee703034c3b91f7c15cae9dae0#info#compaction#21 average throughput is 14.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 08:00:01,244 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/3b036b6d31ed4796bdccfafd4944d03b as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b 2023-09-04 08:00:01,251 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 29b143ee703034c3b91f7c15cae9dae0/info of 29b143ee703034c3b91f7c15cae9dae0 into 3b036b6d31ed4796bdccfafd4944d03b(size=77.1 K), total size for store is 77.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 08:00:01,251 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 08:00:01,251 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0., storeName=29b143ee703034c3b91f7c15cae9dae0/info, priority=13, startTime=1693814401209; duration=0sec 2023-09-04 08:00:01,252 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.ConstantSizeRegionSplitPolicy(109): Should split because info size=77.1 K, sizeToCheck=16.0 K 2023-09-04 08:00:01,252 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.IncreasingToUpperBoundRegionSplitPolicy(84): regionsWithCommonTable=1 2023-09-04 08:00:01,253 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit(227): Splitting TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0., compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:01,253 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:01,254 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45823] assignment.AssignmentManager(1140): Split request from jenkins-hbase4.apache.org,40899,1693814375874, parent={ENCODED => 29b143ee703034c3b91f7c15cae9dae0, NAME => 'TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.', STARTKEY => '', ENDKEY => ''} splitKey=row0064 2023-09-04 08:00:01,262 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:01,268 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45823] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=29b143ee703034c3b91f7c15cae9dae0, daughterA=ad140688bd567031ea0b58fc15034bc1, daughterB=ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:01,269 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=29b143ee703034c3b91f7c15cae9dae0, daughterA=ad140688bd567031ea0b58fc15034bc1, daughterB=ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:01,269 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=29b143ee703034c3b91f7c15cae9dae0, daughterA=ad140688bd567031ea0b58fc15034bc1, daughterB=ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:01,269 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=29b143ee703034c3b91f7c15cae9dae0, daughterA=ad140688bd567031ea0b58fc15034bc1, daughterB=ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:01,277 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=29b143ee703034c3b91f7c15cae9dae0, UNASSIGN}] 2023-09-04 08:00:01,278 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=29b143ee703034c3b91f7c15cae9dae0, UNASSIGN 2023-09-04 08:00:01,279 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=29b143ee703034c3b91f7c15cae9dae0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:01,279 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693814401279"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814401279"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814401279"}]},"ts":"1693814401279"} 2023-09-04 08:00:01,281 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 29b143ee703034c3b91f7c15cae9dae0, server=jenkins-hbase4.apache.org,40899,1693814375874}] 2023-09-04 08:00:01,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:00:01,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 29b143ee703034c3b91f7c15cae9dae0, disabling compactions & flushes 2023-09-04 08:00:01,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 08:00:01,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 08:00:01,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. after waiting 0 ms 2023-09-04 08:00:01,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 08:00:01,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 29b143ee703034c3b91f7c15cae9dae0 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-09-04 08:00:01,452 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=112 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/aa4fae4af49c4a3c869b719476a15c7b 2023-09-04 08:00:01,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.tmp/info/aa4fae4af49c4a3c869b719476a15c7b as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/aa4fae4af49c4a3c869b719476a15c7b 2023-09-04 08:00:01,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/aa4fae4af49c4a3c869b719476a15c7b, entries=21, sequenceid=112, filesize=26.3 K 2023-09-04 08:00:01,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=0 B/0 for 29b143ee703034c3b91f7c15cae9dae0 in 25ms, sequenceid=112, compaction requested=false 2023-09-04 08:00:01,472 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b687bd1d3a4c4b41a5f69e85bb9a8061, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bea6cf3a5a1547c98aa59d2f87a9d34d, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bb6421bcffa24634abd5214adb56d85c, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b686a481b9984fdf906843364a8f0267, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/fa95dc614ed5462ba30d71bd5b7f902f, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/ff919acf66944b58b35cac5c46124a9c] to archive 2023-09-04 08:00:01,473 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 08:00:01,475 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b687bd1d3a4c4b41a5f69e85bb9a8061 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b687bd1d3a4c4b41a5f69e85bb9a8061 2023-09-04 08:00:01,476 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bea6cf3a5a1547c98aa59d2f87a9d34d to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bea6cf3a5a1547c98aa59d2f87a9d34d 2023-09-04 08:00:01,478 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bb6421bcffa24634abd5214adb56d85c to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/bb6421bcffa24634abd5214adb56d85c 2023-09-04 08:00:01,479 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b686a481b9984fdf906843364a8f0267 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/b686a481b9984fdf906843364a8f0267 2023-09-04 08:00:01,481 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/fa95dc614ed5462ba30d71bd5b7f902f to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/fa95dc614ed5462ba30d71bd5b7f902f 2023-09-04 08:00:01,482 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/ff919acf66944b58b35cac5c46124a9c to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/ff919acf66944b58b35cac5c46124a9c 2023-09-04 08:00:01,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/recovered.edits/115.seqid, newMaxSeqId=115, maxSeqId=1 2023-09-04 08:00:01,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. 2023-09-04 08:00:01,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 29b143ee703034c3b91f7c15cae9dae0: 2023-09-04 08:00:01,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:00:01,491 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=29b143ee703034c3b91f7c15cae9dae0, regionState=CLOSED 2023-09-04 08:00:01,492 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693814401491"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814401491"}]},"ts":"1693814401491"} 2023-09-04 08:00:01,495 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=14, resume processing ppid=13 2023-09-04 08:00:01,495 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 29b143ee703034c3b91f7c15cae9dae0, server=jenkins-hbase4.apache.org,40899,1693814375874 in 212 msec 2023-09-04 08:00:01,497 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=13, resume processing ppid=12 2023-09-04 08:00:01,497 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=29b143ee703034c3b91f7c15cae9dae0, UNASSIGN in 218 msec 2023-09-04 08:00:01,510 INFO [PEWorker-4] assignment.SplitTableRegionProcedure(694): pid=12 splitting 2 storefiles, region=29b143ee703034c3b91f7c15cae9dae0, threads=2 2023-09-04 08:00:01,511 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b for region: 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:00:01,511 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/aa4fae4af49c4a3c869b719476a15c7b for region: 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:00:01,527 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/aa4fae4af49c4a3c869b719476a15c7b, top=true 2023-09-04 08:00:01,539 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/.splits/ff7189e0fe31a05d243cdc21b767001a/info/TestLogRolling-testLogRolling=29b143ee703034c3b91f7c15cae9dae0-aa4fae4af49c4a3c869b719476a15c7b for child: ff7189e0fe31a05d243cdc21b767001a, parent: 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:00:01,539 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/aa4fae4af49c4a3c869b719476a15c7b for region: 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:00:01,555 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b for region: 29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:00:01,555 DEBUG [PEWorker-4] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 29b143ee703034c3b91f7c15cae9dae0 Daughter A: 1 storefiles, Daughter B: 2 storefiles. 2023-09-04 08:00:01,590 DEBUG [PEWorker-4] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/recovered.edits/115.seqid, newMaxSeqId=115, maxSeqId=-1 2023-09-04 08:00:01,592 DEBUG [PEWorker-4] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/recovered.edits/115.seqid, newMaxSeqId=115, maxSeqId=-1 2023-09-04 08:00:01,594 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1693814401594"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1693814401594"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1693814401594"}]},"ts":"1693814401594"} 2023-09-04 08:00:01,595 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693814401594"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814401594"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814401594"}]},"ts":"1693814401594"} 2023-09-04 08:00:01,595 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693814401594"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814401594"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814401594"}]},"ts":"1693814401594"} 2023-09-04 08:00:01,636 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40899] regionserver.HRegion(9158): Flush requested on 1588230740 2023-09-04 08:00:01,636 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-09-04 08:00:01,636 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.82 KB heapSize=8.36 KB 2023-09-04 08:00:01,645 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ad140688bd567031ea0b58fc15034bc1, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ff7189e0fe31a05d243cdc21b767001a, ASSIGN}] 2023-09-04 08:00:01,646 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ff7189e0fe31a05d243cdc21b767001a, ASSIGN 2023-09-04 08:00:01,646 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ad140688bd567031ea0b58fc15034bc1, ASSIGN 2023-09-04 08:00:01,647 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ad140688bd567031ea0b58fc15034bc1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40899,1693814375874; forceNewPlan=false, retain=false 2023-09-04 08:00:01,647 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ff7189e0fe31a05d243cdc21b767001a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40899,1693814375874; forceNewPlan=false, retain=false 2023-09-04 08:00:01,653 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.61 KB at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/.tmp/info/4d2467f7561444f8b769854905b233a8 2023-09-04 08:00:01,670 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=216 B at sequenceid=17 (bloomFilter=false), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/.tmp/table/3f3c7c01bc044cf2ad2d6817c16ea72f 2023-09-04 08:00:01,676 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/.tmp/info/4d2467f7561444f8b769854905b233a8 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/info/4d2467f7561444f8b769854905b233a8 2023-09-04 08:00:01,682 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/info/4d2467f7561444f8b769854905b233a8, entries=29, sequenceid=17, filesize=8.6 K 2023-09-04 08:00:01,684 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/.tmp/table/3f3c7c01bc044cf2ad2d6817c16ea72f as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/table/3f3c7c01bc044cf2ad2d6817c16ea72f 2023-09-04 08:00:01,690 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/table/3f3c7c01bc044cf2ad2d6817c16ea72f, entries=4, sequenceid=17, filesize=4.8 K 2023-09-04 08:00:01,692 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.82 KB/4934, heapSize ~8.08 KB/8272, currentSize=0 B/0 for 1588230740 in 56ms, sequenceid=17, compaction requested=false 2023-09-04 08:00:01,692 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-09-04 08:00:01,798 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=ad140688bd567031ea0b58fc15034bc1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:01,798 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=ff7189e0fe31a05d243cdc21b767001a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:01,798 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693814401798"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814401798"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814401798"}]},"ts":"1693814401798"} 2023-09-04 08:00:01,799 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693814401798"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814401798"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814401798"}]},"ts":"1693814401798"} 2023-09-04 08:00:01,801 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure ad140688bd567031ea0b58fc15034bc1, server=jenkins-hbase4.apache.org,40899,1693814375874}] 2023-09-04 08:00:01,801 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure ff7189e0fe31a05d243cdc21b767001a, server=jenkins-hbase4.apache.org,40899,1693814375874}] 2023-09-04 08:00:01,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:00:01,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ad140688bd567031ea0b58fc15034bc1, NAME => 'TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1.', STARTKEY => '', ENDKEY => 'row0064'} 2023-09-04 08:00:01,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling ad140688bd567031ea0b58fc15034bc1 2023-09-04 08:00:01,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 08:00:01,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ad140688bd567031ea0b58fc15034bc1 2023-09-04 08:00:01,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ad140688bd567031ea0b58fc15034bc1 2023-09-04 08:00:01,959 INFO [StoreOpener-ad140688bd567031ea0b58fc15034bc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ad140688bd567031ea0b58fc15034bc1 2023-09-04 08:00:01,960 DEBUG [StoreOpener-ad140688bd567031ea0b58fc15034bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/info 2023-09-04 08:00:01,960 DEBUG [StoreOpener-ad140688bd567031ea0b58fc15034bc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/info 2023-09-04 08:00:01,960 INFO [StoreOpener-ad140688bd567031ea0b58fc15034bc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ad140688bd567031ea0b58fc15034bc1 columnFamilyName info 2023-09-04 08:00:01,975 DEBUG [StoreOpener-ad140688bd567031ea0b58fc15034bc1-1] regionserver.HStore(539): loaded hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0->hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b-bottom 2023-09-04 08:00:01,975 INFO [StoreOpener-ad140688bd567031ea0b58fc15034bc1-1] regionserver.HStore(310): Store=ad140688bd567031ea0b58fc15034bc1/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:00:01,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1 2023-09-04 08:00:01,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1 2023-09-04 08:00:01,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ad140688bd567031ea0b58fc15034bc1 2023-09-04 08:00:01,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ad140688bd567031ea0b58fc15034bc1; next sequenceid=116; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=810256, jitterRate=0.03029458224773407}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 08:00:01,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ad140688bd567031ea0b58fc15034bc1: 2023-09-04 08:00:01,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1., pid=17, masterSystemTime=1693814401953 2023-09-04 08:00:01,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:01,985 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-09-04 08:00:01,986 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:00:01,986 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): ad140688bd567031ea0b58fc15034bc1/info is initiating minor compaction (all files) 2023-09-04 08:00:01,986 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad140688bd567031ea0b58fc15034bc1/info in TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:00:01,986 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0->hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b-bottom] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/.tmp, totalSize=77.1 K 2023-09-04 08:00:01,987 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0, keycount=35, bloomtype=ROW, size=77.1 K, encoding=NONE, compression=NONE, seqNum=87, earliestPutTs=1693814387024 2023-09-04 08:00:01,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:00:01,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:00:01,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:01,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ff7189e0fe31a05d243cdc21b767001a, NAME => 'TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.', STARTKEY => 'row0064', ENDKEY => ''} 2023-09-04 08:00:01,988 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=ad140688bd567031ea0b58fc15034bc1, regionState=OPEN, openSeqNum=116, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:01,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestLogRolling-testLogRolling ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:01,988 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693814401988"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814401988"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814401988"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814401988"}]},"ts":"1693814401988"} 2023-09-04 08:00:01,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 08:00:01,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:01,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:01,993 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=17, resume processing ppid=15 2023-09-04 08:00:01,994 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ad140688bd567031ea0b58fc15034bc1, ASSIGN in 348 msec 2023-09-04 08:00:02,001 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure ad140688bd567031ea0b58fc15034bc1, server=jenkins-hbase4.apache.org,40899,1693814375874 in 190 msec 2023-09-04 08:00:02,001 INFO [StoreOpener-ff7189e0fe31a05d243cdc21b767001a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:02,002 DEBUG [StoreOpener-ff7189e0fe31a05d243cdc21b767001a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info 2023-09-04 08:00:02,003 DEBUG [StoreOpener-ff7189e0fe31a05d243cdc21b767001a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info 2023-09-04 08:00:02,003 INFO [StoreOpener-ff7189e0fe31a05d243cdc21b767001a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff7189e0fe31a05d243cdc21b767001a columnFamilyName info 2023-09-04 08:00:02,004 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad140688bd567031ea0b58fc15034bc1#info#compaction#25 average throughput is 21.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 08:00:02,030 DEBUG [StoreOpener-ff7189e0fe31a05d243cdc21b767001a-1] regionserver.HStore(539): loaded hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0->hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b-top 2023-09-04 08:00:02,035 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/.tmp/info/b2a59b9497974fffa423549841771479 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/info/b2a59b9497974fffa423549841771479 2023-09-04 08:00:02,038 DEBUG [StoreOpener-ff7189e0fe31a05d243cdc21b767001a-1] regionserver.HStore(539): loaded hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/TestLogRolling-testLogRolling=29b143ee703034c3b91f7c15cae9dae0-aa4fae4af49c4a3c869b719476a15c7b 2023-09-04 08:00:02,039 INFO [StoreOpener-ff7189e0fe31a05d243cdc21b767001a-1] regionserver.HStore(310): Store=ff7189e0fe31a05d243cdc21b767001a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:00:02,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:02,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:02,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:02,046 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 1 (all) file(s) in ad140688bd567031ea0b58fc15034bc1/info of ad140688bd567031ea0b58fc15034bc1 into b2a59b9497974fffa423549841771479(size=69.7 K), total size for store is 69.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 08:00:02,046 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad140688bd567031ea0b58fc15034bc1: 2023-09-04 08:00:02,046 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1., storeName=ad140688bd567031ea0b58fc15034bc1/info, priority=15, startTime=1693814401984; duration=0sec 2023-09-04 08:00:02,046 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:02,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ff7189e0fe31a05d243cdc21b767001a; next sequenceid=116; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=879905, jitterRate=0.11885814368724823}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 08:00:02,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:02,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a., pid=18, masterSystemTime=1693814401953 2023-09-04 08:00:02,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:02,048 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-09-04 08:00:02,050 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF info since it belongs to recently split daughter region TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:02,050 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): ff7189e0fe31a05d243cdc21b767001a/info is initiating minor compaction (all files) 2023-09-04 08:00:02,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:02,050 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ff7189e0fe31a05d243cdc21b767001a/info in TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:02,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:02,050 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0->hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b-top, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/TestLogRolling-testLogRolling=29b143ee703034c3b91f7c15cae9dae0-aa4fae4af49c4a3c869b719476a15c7b] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp, totalSize=103.4 K 2023-09-04 08:00:02,051 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=ff7189e0fe31a05d243cdc21b767001a, regionState=OPEN, openSeqNum=116, regionLocation=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:02,051 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1693814402051"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814402051"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814402051"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814402051"}]},"ts":"1693814402051"} 2023-09-04 08:00:02,051 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0, keycount=35, bloomtype=ROW, size=77.1 K, encoding=NONE, compression=NONE, seqNum=88, earliestPutTs=1693814387024 2023-09-04 08:00:02,052 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting TestLogRolling-testLogRolling=29b143ee703034c3b91f7c15cae9dae0-aa4fae4af49c4a3c869b719476a15c7b, keycount=21, bloomtype=ROW, size=26.3 K, encoding=NONE, compression=NONE, seqNum=112, earliestPutTs=1693814401165 2023-09-04 08:00:02,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=18, resume processing ppid=16 2023-09-04 08:00:02,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure ff7189e0fe31a05d243cdc21b767001a, server=jenkins-hbase4.apache.org,40899,1693814375874 in 252 msec 2023-09-04 08:00:02,058 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=16, resume processing ppid=12 2023-09-04 08:00:02,058 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestLogRolling-testLogRolling, region=ff7189e0fe31a05d243cdc21b767001a, ASSIGN in 411 msec 2023-09-04 08:00:02,061 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=TestLogRolling-testLogRolling, parent=29b143ee703034c3b91f7c15cae9dae0, daughterA=ad140688bd567031ea0b58fc15034bc1, daughterB=ff7189e0fe31a05d243cdc21b767001a in 797 msec 2023-09-04 08:00:02,066 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): ff7189e0fe31a05d243cdc21b767001a#info#compaction#26 average throughput is 28.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 08:00:02,087 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/c95fba01904b4a43a2fd9adc827db4f8 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c95fba01904b4a43a2fd9adc827db4f8 2023-09-04 08:00:02,094 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 (all) file(s) in ff7189e0fe31a05d243cdc21b767001a/info of ff7189e0fe31a05d243cdc21b767001a into c95fba01904b4a43a2fd9adc827db4f8(size=33.7 K), total size for store is 33.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 08:00:02,094 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:02,094 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a., storeName=ff7189e0fe31a05d243cdc21b767001a/info, priority=14, startTime=1693814402048; duration=0sec 2023-09-04 08:00:02,095 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:07,036 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-09-04 08:00:11,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] ipc.CallRunner(144): callId: 104 service: ClientService methodName: Mutate size: 1.1 K connection: 172.31.14.131:39458 deadline: 1693814421254, exception=org.apache.hadoop.hbase.NotServingRegionException: TestLogRolling-testLogRolling,,1693814377011.29b143ee703034c3b91f7c15cae9dae0. is not online on jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:33,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:33,281 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 08:00:33,299 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=127 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/667c409624154fed9bf5c6aff4272e29 2023-09-04 08:00:33,306 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/667c409624154fed9bf5c6aff4272e29 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/667c409624154fed9bf5c6aff4272e29 2023-09-04 08:00:33,311 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ff7189e0fe31a05d243cdc21b767001a, server=jenkins-hbase4.apache.org,40899,1693814375874 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 08:00:33,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] ipc.CallRunner(144): callId: 137 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:39458 deadline: 1693814443311, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ff7189e0fe31a05d243cdc21b767001a, server=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:33,312 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/667c409624154fed9bf5c6aff4272e29, entries=8, sequenceid=127, filesize=13.0 K 2023-09-04 08:00:33,314 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for ff7189e0fe31a05d243cdc21b767001a in 33ms, sequenceid=127, compaction requested=false 2023-09-04 08:00:33,314 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:43,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:43,404 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 08:00:43,417 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=152 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/0d5c23f06f77424596b7b134871f093f 2023-09-04 08:00:43,424 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/0d5c23f06f77424596b7b134871f093f as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/0d5c23f06f77424596b7b134871f093f 2023-09-04 08:00:43,429 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/0d5c23f06f77424596b7b134871f093f, entries=22, sequenceid=152, filesize=27.4 K 2023-09-04 08:00:43,430 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for ff7189e0fe31a05d243cdc21b767001a in 26ms, sequenceid=152, compaction requested=true 2023-09-04 08:00:43,430 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:43,431 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:43,431 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 08:00:43,432 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 75856 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 08:00:43,432 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): ff7189e0fe31a05d243cdc21b767001a/info is initiating minor compaction (all files) 2023-09-04 08:00:43,432 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ff7189e0fe31a05d243cdc21b767001a/info in TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:43,432 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c95fba01904b4a43a2fd9adc827db4f8, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/667c409624154fed9bf5c6aff4272e29, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/0d5c23f06f77424596b7b134871f093f] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp, totalSize=74.1 K 2023-09-04 08:00:43,432 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting c95fba01904b4a43a2fd9adc827db4f8, keycount=28, bloomtype=ROW, size=33.7 K, encoding=NONE, compression=NONE, seqNum=112, earliestPutTs=1693814399154 2023-09-04 08:00:43,433 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 667c409624154fed9bf5c6aff4272e29, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=127, earliestPutTs=1693814431272 2023-09-04 08:00:43,433 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 0d5c23f06f77424596b7b134871f093f, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=152, earliestPutTs=1693814433282 2023-09-04 08:00:43,444 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): ff7189e0fe31a05d243cdc21b767001a#info#compaction#29 average throughput is 58.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 08:00:43,461 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/e145cf330b304ee29c802a09bdfe5ac0 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/e145cf330b304ee29c802a09bdfe5ac0 2023-09-04 08:00:43,467 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ff7189e0fe31a05d243cdc21b767001a/info of ff7189e0fe31a05d243cdc21b767001a into e145cf330b304ee29c802a09bdfe5ac0(size=64.7 K), total size for store is 64.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 08:00:43,467 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:43,467 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a., storeName=ff7189e0fe31a05d243cdc21b767001a/info, priority=13, startTime=1693814443430; duration=0sec 2023-09-04 08:00:43,467 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:45,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:45,418 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 08:00:45,433 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=164 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/656f9b625e53432a8be0e021d46d27d9 2023-09-04 08:00:45,440 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/656f9b625e53432a8be0e021d46d27d9 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/656f9b625e53432a8be0e021d46d27d9 2023-09-04 08:00:45,447 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/656f9b625e53432a8be0e021d46d27d9, entries=8, sequenceid=164, filesize=13.0 K 2023-09-04 08:00:45,450 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=20.53 KB/21020 for ff7189e0fe31a05d243cdc21b767001a in 31ms, sequenceid=164, compaction requested=false 2023-09-04 08:00:45,450 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:45,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:45,451 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=21.55 KB heapSize=23.38 KB 2023-09-04 08:00:45,464 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.55 KB at sequenceid=188 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/5767e0a07aa74e17be3fd5d31b2ac3ae 2023-09-04 08:00:45,467 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ff7189e0fe31a05d243cdc21b767001a, server=jenkins-hbase4.apache.org,40899,1693814375874 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 08:00:45,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] ipc.CallRunner(144): callId: 177 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:39458 deadline: 1693814455467, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ff7189e0fe31a05d243cdc21b767001a, server=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:00:45,471 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/5767e0a07aa74e17be3fd5d31b2ac3ae as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/5767e0a07aa74e17be3fd5d31b2ac3ae 2023-09-04 08:00:45,476 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/5767e0a07aa74e17be3fd5d31b2ac3ae, entries=21, sequenceid=188, filesize=26.4 K 2023-09-04 08:00:45,477 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~21.55 KB/22071, heapSize ~23.37 KB/23928, currentSize=8.21 KB/8408 for ff7189e0fe31a05d243cdc21b767001a in 26ms, sequenceid=188, compaction requested=true 2023-09-04 08:00:45,477 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:45,477 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:45,477 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 08:00:45,479 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 106567 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 08:00:45,479 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): ff7189e0fe31a05d243cdc21b767001a/info is initiating minor compaction (all files) 2023-09-04 08:00:45,479 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ff7189e0fe31a05d243cdc21b767001a/info in TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:45,479 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/e145cf330b304ee29c802a09bdfe5ac0, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/656f9b625e53432a8be0e021d46d27d9, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/5767e0a07aa74e17be3fd5d31b2ac3ae] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp, totalSize=104.1 K 2023-09-04 08:00:45,480 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting e145cf330b304ee29c802a09bdfe5ac0, keycount=58, bloomtype=ROW, size=64.7 K, encoding=NONE, compression=NONE, seqNum=152, earliestPutTs=1693814399154 2023-09-04 08:00:45,480 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 656f9b625e53432a8be0e021d46d27d9, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=164, earliestPutTs=1693814443405 2023-09-04 08:00:45,480 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 5767e0a07aa74e17be3fd5d31b2ac3ae, keycount=21, bloomtype=ROW, size=26.4 K, encoding=NONE, compression=NONE, seqNum=188, earliestPutTs=1693814445419 2023-09-04 08:00:45,494 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): ff7189e0fe31a05d243cdc21b767001a#info#compaction#32 average throughput is 29.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 08:00:45,512 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/9fc0c5bfab554f77be28a134fb7090a9 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9fc0c5bfab554f77be28a134fb7090a9 2023-09-04 08:00:45,520 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ff7189e0fe31a05d243cdc21b767001a/info of ff7189e0fe31a05d243cdc21b767001a into 9fc0c5bfab554f77be28a134fb7090a9(size=94.7 K), total size for store is 94.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 08:00:45,520 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:45,520 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a., storeName=ff7189e0fe31a05d243cdc21b767001a/info, priority=13, startTime=1693814445477; duration=0sec 2023-09-04 08:00:45,520 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:55,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:55,518 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=9.24 KB heapSize=10.16 KB 2023-09-04 08:00:55,531 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=9.24 KB at sequenceid=201 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/58f45d8a2f9347ebbf6e34838b5feecd 2023-09-04 08:00:55,537 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/58f45d8a2f9347ebbf6e34838b5feecd as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/58f45d8a2f9347ebbf6e34838b5feecd 2023-09-04 08:00:55,542 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/58f45d8a2f9347ebbf6e34838b5feecd, entries=9, sequenceid=201, filesize=14.0 K 2023-09-04 08:00:55,543 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~9.24 KB/9459, heapSize ~10.15 KB/10392, currentSize=1.03 KB/1051 for ff7189e0fe31a05d243cdc21b767001a in 25ms, sequenceid=201, compaction requested=false 2023-09-04 08:00:55,543 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:57,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:57,530 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 08:00:57,542 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=212 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/33c7a56203db4109ab3d74456704950b 2023-09-04 08:00:57,549 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/33c7a56203db4109ab3d74456704950b as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/33c7a56203db4109ab3d74456704950b 2023-09-04 08:00:57,554 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/33c7a56203db4109ab3d74456704950b, entries=8, sequenceid=212, filesize=13.0 K 2023-09-04 08:00:57,555 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for ff7189e0fe31a05d243cdc21b767001a in 25ms, sequenceid=212, compaction requested=true 2023-09-04 08:00:57,555 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:57,555 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:57,555 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 08:00:57,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:57,556 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 08:00:57,556 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 124627 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 08:00:57,557 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): ff7189e0fe31a05d243cdc21b767001a/info is initiating minor compaction (all files) 2023-09-04 08:00:57,557 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ff7189e0fe31a05d243cdc21b767001a/info in TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:57,557 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9fc0c5bfab554f77be28a134fb7090a9, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/58f45d8a2f9347ebbf6e34838b5feecd, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/33c7a56203db4109ab3d74456704950b] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp, totalSize=121.7 K 2023-09-04 08:00:57,557 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 9fc0c5bfab554f77be28a134fb7090a9, keycount=87, bloomtype=ROW, size=94.7 K, encoding=NONE, compression=NONE, seqNum=188, earliestPutTs=1693814399154 2023-09-04 08:00:57,558 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 58f45d8a2f9347ebbf6e34838b5feecd, keycount=9, bloomtype=ROW, size=14.0 K, encoding=NONE, compression=NONE, seqNum=201, earliestPutTs=1693814445451 2023-09-04 08:00:57,558 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 33c7a56203db4109ab3d74456704950b, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=212, earliestPutTs=1693814455519 2023-09-04 08:00:57,576 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=237 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/c2d11c66a4a14d36b29b715b2bb5a182 2023-09-04 08:00:57,582 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): ff7189e0fe31a05d243cdc21b767001a#info#compaction#36 average throughput is 34.75 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 08:00:57,586 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/c2d11c66a4a14d36b29b715b2bb5a182 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c2d11c66a4a14d36b29b715b2bb5a182 2023-09-04 08:00:57,597 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c2d11c66a4a14d36b29b715b2bb5a182, entries=22, sequenceid=237, filesize=27.4 K 2023-09-04 08:00:57,598 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=3.08 KB/3153 for ff7189e0fe31a05d243cdc21b767001a in 42ms, sequenceid=237, compaction requested=false 2023-09-04 08:00:57,598 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:57,599 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/56f3c43e923640779e5b1e2bcc5cdd93 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/56f3c43e923640779e5b1e2bcc5cdd93 2023-09-04 08:00:57,604 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ff7189e0fe31a05d243cdc21b767001a/info of ff7189e0fe31a05d243cdc21b767001a into 56f3c43e923640779e5b1e2bcc5cdd93(size=112.3 K), total size for store is 139.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 08:00:57,604 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:57,604 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a., storeName=ff7189e0fe31a05d243cdc21b767001a/info, priority=13, startTime=1693814457555; duration=0sec 2023-09-04 08:00:57,604 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:59,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:59,566 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 08:00:59,578 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=249 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/b027f677c25f4eb4b601ebcb8f9502fc 2023-09-04 08:00:59,585 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/b027f677c25f4eb4b601ebcb8f9502fc as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/b027f677c25f4eb4b601ebcb8f9502fc 2023-09-04 08:00:59,591 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/b027f677c25f4eb4b601ebcb8f9502fc, entries=8, sequenceid=249, filesize=13.0 K 2023-09-04 08:00:59,592 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for ff7189e0fe31a05d243cdc21b767001a in 26ms, sequenceid=249, compaction requested=true 2023-09-04 08:00:59,592 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:59,592 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:00:59,592 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 08:00:59,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:00:59,593 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 08:00:59,594 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 156343 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 08:00:59,594 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): ff7189e0fe31a05d243cdc21b767001a/info is initiating minor compaction (all files) 2023-09-04 08:00:59,594 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ff7189e0fe31a05d243cdc21b767001a/info in TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:00:59,594 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/56f3c43e923640779e5b1e2bcc5cdd93, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c2d11c66a4a14d36b29b715b2bb5a182, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/b027f677c25f4eb4b601ebcb8f9502fc] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp, totalSize=152.7 K 2023-09-04 08:00:59,594 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 56f3c43e923640779e5b1e2bcc5cdd93, keycount=104, bloomtype=ROW, size=112.3 K, encoding=NONE, compression=NONE, seqNum=212, earliestPutTs=1693814399154 2023-09-04 08:00:59,595 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting c2d11c66a4a14d36b29b715b2bb5a182, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=237, earliestPutTs=1693814457531 2023-09-04 08:00:59,596 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting b027f677c25f4eb4b601ebcb8f9502fc, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=249, earliestPutTs=1693814457556 2023-09-04 08:00:59,604 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=274 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/bbe2bace44b945629ccf6276a2b8a48e 2023-09-04 08:00:59,610 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): ff7189e0fe31a05d243cdc21b767001a#info#compaction#39 average throughput is 67.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 08:00:59,611 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/bbe2bace44b945629ccf6276a2b8a48e as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/bbe2bace44b945629ccf6276a2b8a48e 2023-09-04 08:00:59,618 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/bbe2bace44b945629ccf6276a2b8a48e, entries=22, sequenceid=274, filesize=27.4 K 2023-09-04 08:00:59,620 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=5.13 KB/5255 for ff7189e0fe31a05d243cdc21b767001a in 27ms, sequenceid=274, compaction requested=false 2023-09-04 08:00:59,620 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:59,628 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/3d9c8e25111c4aaca4a5134dbd4a1884 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3d9c8e25111c4aaca4a5134dbd4a1884 2023-09-04 08:00:59,634 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ff7189e0fe31a05d243cdc21b767001a/info of ff7189e0fe31a05d243cdc21b767001a into 3d9c8e25111c4aaca4a5134dbd4a1884(size=143.5 K), total size for store is 170.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 08:00:59,634 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:00:59,634 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a., storeName=ff7189e0fe31a05d243cdc21b767001a/info, priority=13, startTime=1693814459592; duration=0sec 2023-09-04 08:00:59,634 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:01:01,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:01:01,604 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=8.21 KB heapSize=9.06 KB 2023-09-04 08:01:01,624 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(4965): Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ff7189e0fe31a05d243cdc21b767001a, server=jenkins-hbase4.apache.org,40899,1693814375874 at org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4963) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3155) at org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3006) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2969) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-09-04 08:01:01,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] ipc.CallRunner(144): callId: 269 service: ClientService methodName: Mutate size: 1.2 K connection: 172.31.14.131:39458 deadline: 1693814471624, exception=org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=32.0 K, regionName=ff7189e0fe31a05d243cdc21b767001a, server=jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:01:01,631 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.21 KB at sequenceid=286 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/9f879a5902d441ecad93eab0c3a7405f 2023-09-04 08:01:01,637 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/9f879a5902d441ecad93eab0c3a7405f as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9f879a5902d441ecad93eab0c3a7405f 2023-09-04 08:01:01,641 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9f879a5902d441ecad93eab0c3a7405f, entries=8, sequenceid=286, filesize=13.0 K 2023-09-04 08:01:01,642 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.21 KB/8408, heapSize ~9.05 KB/9264, currentSize=21.55 KB/22071 for ff7189e0fe31a05d243cdc21b767001a in 39ms, sequenceid=286, compaction requested=true 2023-09-04 08:01:01,642 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:01:01,642 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: MemStoreFlusher.0; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:01:01,642 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-09-04 08:01:01,643 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 188264 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-09-04 08:01:01,643 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1912): ff7189e0fe31a05d243cdc21b767001a/info is initiating minor compaction (all files) 2023-09-04 08:01:01,643 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ff7189e0fe31a05d243cdc21b767001a/info in TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:01:01,644 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3d9c8e25111c4aaca4a5134dbd4a1884, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/bbe2bace44b945629ccf6276a2b8a48e, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9f879a5902d441ecad93eab0c3a7405f] into tmpdir=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp, totalSize=183.9 K 2023-09-04 08:01:01,644 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 3d9c8e25111c4aaca4a5134dbd4a1884, keycount=134, bloomtype=ROW, size=143.5 K, encoding=NONE, compression=NONE, seqNum=249, earliestPutTs=1693814399154 2023-09-04 08:01:01,644 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting bbe2bace44b945629ccf6276a2b8a48e, keycount=22, bloomtype=ROW, size=27.4 K, encoding=NONE, compression=NONE, seqNum=274, earliestPutTs=1693814459566 2023-09-04 08:01:01,645 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] compactions.Compactor(207): Compacting 9f879a5902d441ecad93eab0c3a7405f, keycount=8, bloomtype=ROW, size=13.0 K, encoding=NONE, compression=NONE, seqNum=286, earliestPutTs=1693814459594 2023-09-04 08:01:01,656 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] throttle.PressureAwareThroughputController(145): ff7189e0fe31a05d243cdc21b767001a#info#compaction#41 average throughput is 54.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-09-04 08:01:01,668 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/98f3a920c05e4f68857ebbb3c6f21e5b as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/98f3a920c05e4f68857ebbb3c6f21e5b 2023-09-04 08:01:01,673 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ff7189e0fe31a05d243cdc21b767001a/info of ff7189e0fe31a05d243cdc21b767001a into 98f3a920c05e4f68857ebbb3c6f21e5b(size=174.4 K), total size for store is 174.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-09-04 08:01:01,673 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:01:01,673 INFO [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a., storeName=ff7189e0fe31a05d243cdc21b767001a/info, priority=13, startTime=1693814461642; duration=0sec 2023-09-04 08:01:01,673 DEBUG [RS:0;jenkins-hbase4:40899-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-09-04 08:01:11,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40899] regionserver.HRegion(9158): Flush requested on ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:01:11,698 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=22.58 KB heapSize=24.48 KB 2023-09-04 08:01:11,712 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=22.58 KB at sequenceid=312 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/6c733f6c59864eaeb1ed1ab1a23c2f7c 2023-09-04 08:01:11,718 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/6c733f6c59864eaeb1ed1ab1a23c2f7c as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/6c733f6c59864eaeb1ed1ab1a23c2f7c 2023-09-04 08:01:11,724 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/6c733f6c59864eaeb1ed1ab1a23c2f7c, entries=22, sequenceid=312, filesize=27.4 K 2023-09-04 08:01:11,724 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~22.58 KB/23122, heapSize ~24.47 KB/25056, currentSize=7.18 KB/7357 for ff7189e0fe31a05d243cdc21b767001a in 26ms, sequenceid=312, compaction requested=false 2023-09-04 08:01:11,725 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:01:13,705 INFO [Listener at localhost/41609] wal.AbstractTestLogRolling(188): after writing there are 0 log files 2023-09-04 08:01:13,722 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK] 2023-09-04 08:01:13,722 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK] 2023-09-04 08:01:13,725 INFO [Listener at localhost/41609] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814376270 with entries=306, filesize=299.57 KB; new WAL /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814473706 2023-09-04 08:01:13,726 DEBUG [Listener at localhost/41609] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK], DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK]] 2023-09-04 08:01:13,726 DEBUG [Listener at localhost/41609] wal.AbstractFSWAL(716): hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814376270 is not closed yet, will try archiving it next time 2023-09-04 08:01:13,731 INFO [Listener at localhost/41609] regionserver.HRegion(2745): Flushing 3744a879ceb8a9f6ad86aeb3b6fb4bbf 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-04 08:01:13,743 INFO [Listener at localhost/41609] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf/.tmp/info/2f8289f4dd4d4a10bc23360397d035ef 2023-09-04 08:01:13,749 DEBUG [Listener at localhost/41609] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf/.tmp/info/2f8289f4dd4d4a10bc23360397d035ef as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf/info/2f8289f4dd4d4a10bc23360397d035ef 2023-09-04 08:01:13,754 INFO [Listener at localhost/41609] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf/info/2f8289f4dd4d4a10bc23360397d035ef, entries=2, sequenceid=6, filesize=4.8 K 2023-09-04 08:01:13,755 INFO [Listener at localhost/41609] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 3744a879ceb8a9f6ad86aeb3b6fb4bbf in 24ms, sequenceid=6, compaction requested=false 2023-09-04 08:01:13,756 DEBUG [Listener at localhost/41609] regionserver.HRegion(2446): Flush status journal for 3744a879ceb8a9f6ad86aeb3b6fb4bbf: 2023-09-04 08:01:13,756 DEBUG [Listener at localhost/41609] regionserver.HRegion(2446): Flush status journal for ad140688bd567031ea0b58fc15034bc1: 2023-09-04 08:01:13,756 INFO [Listener at localhost/41609] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.26 KB heapSize=4.19 KB 2023-09-04 08:01:13,768 INFO [Listener at localhost/41609] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.26 KB at sequenceid=24 (bloomFilter=false), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/.tmp/info/16b25eb3aec44125954907ba9e363f27 2023-09-04 08:01:13,777 DEBUG [Listener at localhost/41609] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/.tmp/info/16b25eb3aec44125954907ba9e363f27 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/info/16b25eb3aec44125954907ba9e363f27 2023-09-04 08:01:13,782 INFO [Listener at localhost/41609] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/info/16b25eb3aec44125954907ba9e363f27, entries=16, sequenceid=24, filesize=7.0 K 2023-09-04 08:01:13,783 INFO [Listener at localhost/41609] regionserver.HRegion(2948): Finished flush of dataSize ~2.26 KB/2312, heapSize ~3.67 KB/3760, currentSize=0 B/0 for 1588230740 in 27ms, sequenceid=24, compaction requested=false 2023-09-04 08:01:13,783 DEBUG [Listener at localhost/41609] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-09-04 08:01:13,783 INFO [Listener at localhost/41609] regionserver.HRegion(2745): Flushing ff7189e0fe31a05d243cdc21b767001a 1/1 column families, dataSize=7.18 KB heapSize=7.96 KB 2023-09-04 08:01:13,795 INFO [Listener at localhost/41609] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=7.18 KB at sequenceid=322 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/2b3676b7ab494596841e9b43066e1a9b 2023-09-04 08:01:13,800 DEBUG [Listener at localhost/41609] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/.tmp/info/2b3676b7ab494596841e9b43066e1a9b as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/2b3676b7ab494596841e9b43066e1a9b 2023-09-04 08:01:13,805 INFO [Listener at localhost/41609] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/2b3676b7ab494596841e9b43066e1a9b, entries=7, sequenceid=322, filesize=12.0 K 2023-09-04 08:01:13,806 INFO [Listener at localhost/41609] regionserver.HRegion(2948): Finished flush of dataSize ~7.18 KB/7357, heapSize ~7.95 KB/8136, currentSize=0 B/0 for ff7189e0fe31a05d243cdc21b767001a in 23ms, sequenceid=322, compaction requested=true 2023-09-04 08:01:13,806 DEBUG [Listener at localhost/41609] regionserver.HRegion(2446): Flush status journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:01:13,820 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK] 2023-09-04 08:01:13,820 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK] 2023-09-04 08:01:13,823 INFO [Listener at localhost/41609] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814473706 with entries=4, filesize=1.22 KB; new WAL /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814473806 2023-09-04 08:01:13,823 DEBUG [Listener at localhost/41609] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43733,DS-6d705145-03fa-43a7-9ae4-ecae9c98215a,DISK], DatanodeInfoWithStorage[127.0.0.1:40945,DS-4cd26429-99e6-4591-ba18-6922364ff8e1,DISK]] 2023-09-04 08:01:13,823 DEBUG [Listener at localhost/41609] wal.AbstractFSWAL(716): hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814473706 is not closed yet, will try archiving it next time 2023-09-04 08:01:13,824 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814376270 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/oldWALs/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814376270 2023-09-04 08:01:13,824 INFO [Listener at localhost/41609] hbase.Waiter(180): Waiting up to [5,000] milli-secs(wait.for.ratio=[1]) 2023-09-04 08:01:13,826 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/WALs/jenkins-hbase4.apache.org,40899,1693814375874/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814473706 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/oldWALs/jenkins-hbase4.apache.org%2C40899%2C1693814375874.1693814473706 2023-09-04 08:01:13,925 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-04 08:01:13,925 INFO [Listener at localhost/41609] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-09-04 08:01:13,925 DEBUG [Listener at localhost/41609] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x369b80bf to 127.0.0.1:62260 2023-09-04 08:01:13,925 DEBUG [Listener at localhost/41609] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:13,925 DEBUG [Listener at localhost/41609] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-04 08:01:13,925 DEBUG [Listener at localhost/41609] util.JVMClusterUtil(257): Found active master hash=426902765, stopped=false 2023-09-04 08:01:13,925 INFO [Listener at localhost/41609] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 08:01:13,929 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 08:01:13,929 INFO [Listener at localhost/41609] procedure2.ProcedureExecutor(628): Stopping 2023-09-04 08:01:13,929 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:13,929 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 08:01:13,930 DEBUG [Listener at localhost/41609] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x59250f99 to 127.0.0.1:62260 2023-09-04 08:01:13,930 DEBUG [Listener at localhost/41609] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:13,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 08:01:13,930 INFO [Listener at localhost/41609] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,40899,1693814375874' ***** 2023-09-04 08:01:13,930 INFO [Listener at localhost/41609] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-04 08:01:13,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 08:01:13,930 INFO [RS:0;jenkins-hbase4:40899] regionserver.HeapMemoryManager(220): Stopping 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-04 08:01:13,931 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(3305): Received CLOSE for 3744a879ceb8a9f6ad86aeb3b6fb4bbf 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(3305): Received CLOSE for ad140688bd567031ea0b58fc15034bc1 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(3305): Received CLOSE for ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:01:13,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3744a879ceb8a9f6ad86aeb3b6fb4bbf, disabling compactions & flushes 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:01:13,931 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 08:01:13,931 DEBUG [RS:0;jenkins-hbase4:40899] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2cc9af44 to 127.0.0.1:62260 2023-09-04 08:01:13,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 08:01:13,931 DEBUG [RS:0;jenkins-hbase4:40899] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:13,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. after waiting 0 ms 2023-09-04 08:01:13,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-04 08:01:13,931 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-04 08:01:13,932 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-09-04 08:01:13,932 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1478): Online Regions={3744a879ceb8a9f6ad86aeb3b6fb4bbf=hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf., ad140688bd567031ea0b58fc15034bc1=TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1., 1588230740=hbase:meta,,1.1588230740, ff7189e0fe31a05d243cdc21b767001a=TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.} 2023-09-04 08:01:13,932 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 08:01:13,932 DEBUG [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1504): Waiting on 1588230740, 3744a879ceb8a9f6ad86aeb3b6fb4bbf, ad140688bd567031ea0b58fc15034bc1, ff7189e0fe31a05d243cdc21b767001a 2023-09-04 08:01:13,932 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 08:01:13,933 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 08:01:13,933 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 08:01:13,933 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 08:01:13,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/namespace/3744a879ceb8a9f6ad86aeb3b6fb4bbf/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 08:01:13,940 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=1 2023-09-04 08:01:13,940 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 08:01:13,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3744a879ceb8a9f6ad86aeb3b6fb4bbf: 2023-09-04 08:01:13,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693814376500.3744a879ceb8a9f6ad86aeb3b6fb4bbf. 2023-09-04 08:01:13,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ad140688bd567031ea0b58fc15034bc1, disabling compactions & flushes 2023-09-04 08:01:13,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:01:13,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:01:13,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. after waiting 0 ms 2023-09-04 08:01:13,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:01:13,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 08:01:13,941 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 08:01:13,942 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 08:01:13,942 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-04 08:01:13,942 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0->hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b-bottom] to archive 2023-09-04 08:01:13,942 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 08:01:13,944 DEBUG [StoreCloser-TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:01:13,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ad140688bd567031ea0b58fc15034bc1/recovered.edits/120.seqid, newMaxSeqId=120, maxSeqId=115 2023-09-04 08:01:13,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:01:13,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ad140688bd567031ea0b58fc15034bc1: 2023-09-04 08:01:13,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,,1693814401262.ad140688bd567031ea0b58fc15034bc1. 2023-09-04 08:01:13,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ff7189e0fe31a05d243cdc21b767001a, disabling compactions & flushes 2023-09-04 08:01:13,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:01:13,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:01:13,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. after waiting 0 ms 2023-09-04 08:01:13,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:01:13,959 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0->hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/29b143ee703034c3b91f7c15cae9dae0/info/3b036b6d31ed4796bdccfafd4944d03b-top, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c95fba01904b4a43a2fd9adc827db4f8, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/TestLogRolling-testLogRolling=29b143ee703034c3b91f7c15cae9dae0-aa4fae4af49c4a3c869b719476a15c7b, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/667c409624154fed9bf5c6aff4272e29, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/e145cf330b304ee29c802a09bdfe5ac0, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/0d5c23f06f77424596b7b134871f093f, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/656f9b625e53432a8be0e021d46d27d9, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9fc0c5bfab554f77be28a134fb7090a9, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/5767e0a07aa74e17be3fd5d31b2ac3ae, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/58f45d8a2f9347ebbf6e34838b5feecd, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/56f3c43e923640779e5b1e2bcc5cdd93, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/33c7a56203db4109ab3d74456704950b, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c2d11c66a4a14d36b29b715b2bb5a182, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3d9c8e25111c4aaca4a5134dbd4a1884, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/b027f677c25f4eb4b601ebcb8f9502fc, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/bbe2bace44b945629ccf6276a2b8a48e, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9f879a5902d441ecad93eab0c3a7405f] to archive 2023-09-04 08:01:13,960 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-09-04 08:01:13,962 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3b036b6d31ed4796bdccfafd4944d03b.29b143ee703034c3b91f7c15cae9dae0 2023-09-04 08:01:13,963 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c95fba01904b4a43a2fd9adc827db4f8 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c95fba01904b4a43a2fd9adc827db4f8 2023-09-04 08:01:13,965 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/TestLogRolling-testLogRolling=29b143ee703034c3b91f7c15cae9dae0-aa4fae4af49c4a3c869b719476a15c7b to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/TestLogRolling-testLogRolling=29b143ee703034c3b91f7c15cae9dae0-aa4fae4af49c4a3c869b719476a15c7b 2023-09-04 08:01:13,966 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/667c409624154fed9bf5c6aff4272e29 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/667c409624154fed9bf5c6aff4272e29 2023-09-04 08:01:13,967 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/e145cf330b304ee29c802a09bdfe5ac0 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/e145cf330b304ee29c802a09bdfe5ac0 2023-09-04 08:01:13,968 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/0d5c23f06f77424596b7b134871f093f to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/0d5c23f06f77424596b7b134871f093f 2023-09-04 08:01:13,969 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/656f9b625e53432a8be0e021d46d27d9 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/656f9b625e53432a8be0e021d46d27d9 2023-09-04 08:01:13,970 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9fc0c5bfab554f77be28a134fb7090a9 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9fc0c5bfab554f77be28a134fb7090a9 2023-09-04 08:01:13,971 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/5767e0a07aa74e17be3fd5d31b2ac3ae to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/5767e0a07aa74e17be3fd5d31b2ac3ae 2023-09-04 08:01:13,972 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/58f45d8a2f9347ebbf6e34838b5feecd to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/58f45d8a2f9347ebbf6e34838b5feecd 2023-09-04 08:01:13,974 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/56f3c43e923640779e5b1e2bcc5cdd93 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/56f3c43e923640779e5b1e2bcc5cdd93 2023-09-04 08:01:13,975 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/33c7a56203db4109ab3d74456704950b to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/33c7a56203db4109ab3d74456704950b 2023-09-04 08:01:13,976 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c2d11c66a4a14d36b29b715b2bb5a182 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/c2d11c66a4a14d36b29b715b2bb5a182 2023-09-04 08:01:13,977 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3d9c8e25111c4aaca4a5134dbd4a1884 to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/3d9c8e25111c4aaca4a5134dbd4a1884 2023-09-04 08:01:13,978 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/b027f677c25f4eb4b601ebcb8f9502fc to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/b027f677c25f4eb4b601ebcb8f9502fc 2023-09-04 08:01:13,979 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/bbe2bace44b945629ccf6276a2b8a48e to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/bbe2bace44b945629ccf6276a2b8a48e 2023-09-04 08:01:13,980 DEBUG [StoreCloser-TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9f879a5902d441ecad93eab0c3a7405f to hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/archive/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/info/9f879a5902d441ecad93eab0c3a7405f 2023-09-04 08:01:13,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/data/default/TestLogRolling-testLogRolling/ff7189e0fe31a05d243cdc21b767001a/recovered.edits/325.seqid, newMaxSeqId=325, maxSeqId=115 2023-09-04 08:01:13,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:01:13,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ff7189e0fe31a05d243cdc21b767001a: 2023-09-04 08:01:13,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed TestLogRolling-testLogRolling,row0064,1693814401262.ff7189e0fe31a05d243cdc21b767001a. 2023-09-04 08:01:14,133 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,40899,1693814375874; all regions closed. 2023-09-04 08:01:14,140 DEBUG [RS:0;jenkins-hbase4:40899] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/oldWALs 2023-09-04 08:01:14,140 INFO [RS:0;jenkins-hbase4:40899] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40899%2C1693814375874.meta:.meta(num 1693814376435) 2023-09-04 08:01:14,141 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-09-04 08:01:14,142 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-09-04 08:01:14,144 DEBUG [RS:0;jenkins-hbase4:40899] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/oldWALs 2023-09-04 08:01:14,144 INFO [RS:0;jenkins-hbase4:40899] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40899%2C1693814375874:(num 1693814473806) 2023-09-04 08:01:14,144 DEBUG [RS:0;jenkins-hbase4:40899] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:14,144 INFO [RS:0;jenkins-hbase4:40899] regionserver.LeaseManager(133): Closed leases 2023-09-04 08:01:14,144 INFO [RS:0;jenkins-hbase4:40899] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-09-04 08:01:14,144 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 08:01:14,145 INFO [RS:0;jenkins-hbase4:40899] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40899 2023-09-04 08:01:14,146 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-04 08:01:14,148 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40899,1693814375874 2023-09-04 08:01:14,148 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 08:01:14,148 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 08:01:14,149 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40899,1693814375874] 2023-09-04 08:01:14,149 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40899,1693814375874; numProcessing=1 2023-09-04 08:01:14,150 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40899,1693814375874 already deleted, retry=false 2023-09-04 08:01:14,151 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40899,1693814375874 expired; onlineServers=0 2023-09-04 08:01:14,151 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45823,1693814375809' ***** 2023-09-04 08:01:14,151 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-04 08:01:14,151 DEBUG [M:0;jenkins-hbase4:45823] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d4d931b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 08:01:14,151 INFO [M:0;jenkins-hbase4:45823] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45823,1693814375809 2023-09-04 08:01:14,151 INFO [M:0;jenkins-hbase4:45823] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45823,1693814375809; all regions closed. 2023-09-04 08:01:14,151 DEBUG [M:0;jenkins-hbase4:45823] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:14,151 DEBUG [M:0;jenkins-hbase4:45823] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-04 08:01:14,151 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-04 08:01:14,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814376062] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814376062,5,FailOnTimeoutGroup] 2023-09-04 08:01:14,151 DEBUG [M:0;jenkins-hbase4:45823] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-04 08:01:14,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814376062] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814376062,5,FailOnTimeoutGroup] 2023-09-04 08:01:14,152 INFO [M:0;jenkins-hbase4:45823] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-04 08:01:14,152 INFO [M:0;jenkins-hbase4:45823] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-04 08:01:14,152 INFO [M:0;jenkins-hbase4:45823] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-04 08:01:14,152 DEBUG [M:0;jenkins-hbase4:45823] master.HMaster(1512): Stopping service threads 2023-09-04 08:01:14,152 INFO [M:0;jenkins-hbase4:45823] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-04 08:01:14,153 ERROR [M:0;jenkins-hbase4:45823] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-09-04 08:01:14,153 INFO [M:0;jenkins-hbase4:45823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-04 08:01:14,153 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-04 08:01:14,154 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-04 08:01:14,154 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:14,154 DEBUG [M:0;jenkins-hbase4:45823] zookeeper.ZKUtil(398): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-04 08:01:14,154 WARN [M:0;jenkins-hbase4:45823] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-04 08:01:14,154 INFO [M:0;jenkins-hbase4:45823] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-04 08:01:14,154 INFO [M:0;jenkins-hbase4:45823] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-04 08:01:14,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 08:01:14,155 DEBUG [M:0;jenkins-hbase4:45823] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 08:01:14,155 INFO [M:0;jenkins-hbase4:45823] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:14,155 DEBUG [M:0;jenkins-hbase4:45823] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:14,155 DEBUG [M:0;jenkins-hbase4:45823] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 08:01:14,155 DEBUG [M:0;jenkins-hbase4:45823] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:14,155 INFO [M:0;jenkins-hbase4:45823] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=64.71 KB heapSize=78.42 KB 2023-09-04 08:01:14,167 INFO [M:0;jenkins-hbase4:45823] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=64.71 KB at sequenceid=160 (bloomFilter=true), to=hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3bdcaeeab8e24ddb875c5675dca65058 2023-09-04 08:01:14,172 INFO [M:0;jenkins-hbase4:45823] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3bdcaeeab8e24ddb875c5675dca65058 2023-09-04 08:01:14,173 DEBUG [M:0;jenkins-hbase4:45823] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3bdcaeeab8e24ddb875c5675dca65058 as hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3bdcaeeab8e24ddb875c5675dca65058 2023-09-04 08:01:14,178 INFO [M:0;jenkins-hbase4:45823] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3bdcaeeab8e24ddb875c5675dca65058 2023-09-04 08:01:14,178 INFO [M:0;jenkins-hbase4:45823] regionserver.HStore(1080): Added hdfs://localhost:40219/user/jenkins/test-data/d5ad7315-cb5f-8eb8-aa02-16d49104b318/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3bdcaeeab8e24ddb875c5675dca65058, entries=18, sequenceid=160, filesize=6.9 K 2023-09-04 08:01:14,178 INFO [M:0;jenkins-hbase4:45823] regionserver.HRegion(2948): Finished flush of dataSize ~64.71 KB/66268, heapSize ~78.41 KB/80288, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=160, compaction requested=false 2023-09-04 08:01:14,180 INFO [M:0;jenkins-hbase4:45823] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:14,180 DEBUG [M:0;jenkins-hbase4:45823] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 08:01:14,182 INFO [M:0;jenkins-hbase4:45823] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-04 08:01:14,182 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 08:01:14,182 INFO [M:0;jenkins-hbase4:45823] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45823 2023-09-04 08:01:14,184 DEBUG [M:0;jenkins-hbase4:45823] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,45823,1693814375809 already deleted, retry=false 2023-09-04 08:01:14,250 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 08:01:14,250 INFO [RS:0;jenkins-hbase4:40899] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,40899,1693814375874; zookeeper connection closed. 2023-09-04 08:01:14,250 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): regionserver:40899-0x10075f37d0b0001, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 08:01:14,250 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@710628df] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@710628df 2023-09-04 08:01:14,251 INFO [Listener at localhost/41609] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-04 08:01:14,350 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 08:01:14,350 INFO [M:0;jenkins-hbase4:45823] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45823,1693814375809; zookeeper connection closed. 2023-09-04 08:01:14,350 DEBUG [Listener at localhost/41609-EventThread] zookeeper.ZKWatcher(600): master:45823-0x10075f37d0b0000, quorum=127.0.0.1:62260, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 08:01:14,351 WARN [Listener at localhost/41609] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 08:01:14,354 INFO [Listener at localhost/41609] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 08:01:14,460 WARN [BP-1940953692-172.31.14.131-1693814375252 heartbeating to localhost/127.0.0.1:40219] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 08:01:14,460 WARN [BP-1940953692-172.31.14.131-1693814375252 heartbeating to localhost/127.0.0.1:40219] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1940953692-172.31.14.131-1693814375252 (Datanode Uuid 9a39d184-686f-466b-8c3c-16e4016500f0) service to localhost/127.0.0.1:40219 2023-09-04 08:01:14,460 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/cluster_de6439f2-5638-761f-c571-fce95f32cc03/dfs/data/data3/current/BP-1940953692-172.31.14.131-1693814375252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 08:01:14,461 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/cluster_de6439f2-5638-761f-c571-fce95f32cc03/dfs/data/data4/current/BP-1940953692-172.31.14.131-1693814375252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 08:01:14,462 WARN [Listener at localhost/41609] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 08:01:14,465 INFO [Listener at localhost/41609] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 08:01:14,569 WARN [BP-1940953692-172.31.14.131-1693814375252 heartbeating to localhost/127.0.0.1:40219] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 08:01:14,569 WARN [BP-1940953692-172.31.14.131-1693814375252 heartbeating to localhost/127.0.0.1:40219] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1940953692-172.31.14.131-1693814375252 (Datanode Uuid c777e33f-9737-4700-8f48-dd1b2d25cda9) service to localhost/127.0.0.1:40219 2023-09-04 08:01:14,570 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/cluster_de6439f2-5638-761f-c571-fce95f32cc03/dfs/data/data1/current/BP-1940953692-172.31.14.131-1693814375252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 08:01:14,570 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/cluster_de6439f2-5638-761f-c571-fce95f32cc03/dfs/data/data2/current/BP-1940953692-172.31.14.131-1693814375252] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 08:01:14,582 INFO [Listener at localhost/41609] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 08:01:14,698 INFO [Listener at localhost/41609] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-04 08:01:14,729 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-04 08:01:14,738 INFO [Listener at localhost/41609] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRolling Thread=75 (was 64) - Thread LEAK? -, OpenFileDescriptor=464 (was 442) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=56 (was 83), ProcessCount=167 (was 167), AvailableMemoryMB=8197 (was 8447) 2023-09-04 08:01:14,746 INFO [Listener at localhost/41609] hbase.ResourceChecker(147): before: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=76, OpenFileDescriptor=464, MaxFileDescriptor=60000, SystemLoadAverage=56, ProcessCount=167, AvailableMemoryMB=8197 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.log.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/hadoop.log.dir so I do NOT create it in target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(445): System.getProperty("hadoop.tmp.dir") already set to: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ad0fa84b-c42c-6426-289a-cf2f7e51f232/hadoop.tmp.dir so I do NOT create it in target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/cluster_7d658e7b-cc6f-6476-4fa5-eba95b181060, deleteOnExit=true 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/test.cache.data in system properties and HBase conf 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/hadoop.tmp.dir in system properties and HBase conf 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/hadoop.log.dir in system properties and HBase conf 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/mapreduce.cluster.local.dir in system properties and HBase conf 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-09-04 08:01:14,747 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-09-04 08:01:14,748 DEBUG [Listener at localhost/41609] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-04 08:01:14,748 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-09-04 08:01:14,748 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-09-04 08:01:14,748 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-09-04 08:01:14,748 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 08:01:14,748 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-09-04 08:01:14,748 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-09-04 08:01:14,748 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-09-04 08:01:14,748 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 08:01:14,749 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-09-04 08:01:14,749 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/nfs.dump.dir in system properties and HBase conf 2023-09-04 08:01:14,749 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/java.io.tmpdir in system properties and HBase conf 2023-09-04 08:01:14,749 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-09-04 08:01:14,749 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-09-04 08:01:14,749 INFO [Listener at localhost/41609] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-09-04 08:01:14,754 WARN [Listener at localhost/41609] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 08:01:14,754 WARN [Listener at localhost/41609] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 08:01:14,790 WARN [Listener at localhost/41609] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 08:01:14,792 INFO [Listener at localhost/41609] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 08:01:14,796 INFO [Listener at localhost/41609] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/java.io.tmpdir/Jetty_localhost_34917_hdfs____467x6w/webapp 2023-09-04 08:01:14,888 INFO [Listener at localhost/41609] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34917 2023-09-04 08:01:14,892 WARN [Listener at localhost/41609] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-09-04 08:01:14,892 WARN [Listener at localhost/41609] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-09-04 08:01:14,934 WARN [Listener at localhost/33613] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 08:01:14,944 WARN [Listener at localhost/33613] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 08:01:14,946 WARN [Listener at localhost/33613] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 08:01:14,947 INFO [Listener at localhost/33613] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 08:01:14,952 INFO [Listener at localhost/33613] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/java.io.tmpdir/Jetty_localhost_43485_datanode____.3m0kav/webapp 2023-09-04 08:01:15,043 INFO [Listener at localhost/33613] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43485 2023-09-04 08:01:15,052 WARN [Listener at localhost/44667] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 08:01:15,064 WARN [Listener at localhost/44667] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-09-04 08:01:15,066 WARN [Listener at localhost/44667] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-09-04 08:01:15,068 INFO [Listener at localhost/44667] log.Slf4jLog(67): jetty-6.1.26 2023-09-04 08:01:15,073 INFO [Listener at localhost/44667] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/java.io.tmpdir/Jetty_localhost_42415_datanode____.7o6yv3/webapp 2023-09-04 08:01:15,151 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x92b818f8224cd34: Processing first storage report for DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621 from datanode d42ac892-09b9-4145-b42e-55e763d7a8d2 2023-09-04 08:01:15,151 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x92b818f8224cd34: from storage DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621 node DatanodeRegistration(127.0.0.1:32899, datanodeUuid=d42ac892-09b9-4145-b42e-55e763d7a8d2, infoPort=42603, infoSecurePort=0, ipcPort=44667, storageInfo=lv=-57;cid=testClusterID;nsid=815617731;c=1693814474757), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 08:01:15,151 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x92b818f8224cd34: Processing first storage report for DS-3ea00f8f-50c9-4446-9d09-995924765a04 from datanode d42ac892-09b9-4145-b42e-55e763d7a8d2 2023-09-04 08:01:15,151 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x92b818f8224cd34: from storage DS-3ea00f8f-50c9-4446-9d09-995924765a04 node DatanodeRegistration(127.0.0.1:32899, datanodeUuid=d42ac892-09b9-4145-b42e-55e763d7a8d2, infoPort=42603, infoSecurePort=0, ipcPort=44667, storageInfo=lv=-57;cid=testClusterID;nsid=815617731;c=1693814474757), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 08:01:15,178 INFO [Listener at localhost/44667] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42415 2023-09-04 08:01:15,186 WARN [Listener at localhost/34091] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-09-04 08:01:15,280 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5ee323da51ca2969: Processing first storage report for DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2 from datanode e42dc467-e8c1-4a4a-a472-fe6bd944c9cc 2023-09-04 08:01:15,280 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5ee323da51ca2969: from storage DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2 node DatanodeRegistration(127.0.0.1:43333, datanodeUuid=e42dc467-e8c1-4a4a-a472-fe6bd944c9cc, infoPort=42399, infoSecurePort=0, ipcPort=34091, storageInfo=lv=-57;cid=testClusterID;nsid=815617731;c=1693814474757), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 08:01:15,280 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x5ee323da51ca2969: Processing first storage report for DS-a176fb51-983b-4664-a324-60368ffc3f9c from datanode e42dc467-e8c1-4a4a-a472-fe6bd944c9cc 2023-09-04 08:01:15,280 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x5ee323da51ca2969: from storage DS-a176fb51-983b-4664-a324-60368ffc3f9c node DatanodeRegistration(127.0.0.1:43333, datanodeUuid=e42dc467-e8c1-4a4a-a472-fe6bd944c9cc, infoPort=42399, infoSecurePort=0, ipcPort=34091, storageInfo=lv=-57;cid=testClusterID;nsid=815617731;c=1693814474757), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-09-04 08:01:15,293 DEBUG [Listener at localhost/34091] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8 2023-09-04 08:01:15,295 INFO [Listener at localhost/34091] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/cluster_7d658e7b-cc6f-6476-4fa5-eba95b181060/zookeeper_0, clientPort=54397, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/cluster_7d658e7b-cc6f-6476-4fa5-eba95b181060/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/cluster_7d658e7b-cc6f-6476-4fa5-eba95b181060/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-09-04 08:01:15,296 INFO [Listener at localhost/34091] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=54397 2023-09-04 08:01:15,297 INFO [Listener at localhost/34091] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 08:01:15,298 INFO [Listener at localhost/34091] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 08:01:15,313 INFO [Listener at localhost/34091] util.FSUtils(471): Created version file at hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8 with version=8 2023-09-04 08:01:15,313 INFO [Listener at localhost/34091] hbase.HBaseTestingUtility(1408): The hbase.fs.tmp.dir is set to hdfs://localhost:39519/user/jenkins/test-data/bf27cfa2-acf3-c8f2-e3a6-bc3bb26bbedb/hbase-staging 2023-09-04 08:01:15,315 INFO [Listener at localhost/34091] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 08:01:15,316 INFO [Listener at localhost/34091] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 08:01:15,316 INFO [Listener at localhost/34091] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 08:01:15,316 INFO [Listener at localhost/34091] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 08:01:15,316 INFO [Listener at localhost/34091] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 08:01:15,316 INFO [Listener at localhost/34091] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 08:01:15,316 INFO [Listener at localhost/34091] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 08:01:15,317 INFO [Listener at localhost/34091] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43721 2023-09-04 08:01:15,318 INFO [Listener at localhost/34091] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 08:01:15,319 INFO [Listener at localhost/34091] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 08:01:15,321 INFO [Listener at localhost/34091] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43721 connecting to ZooKeeper ensemble=127.0.0.1:54397 2023-09-04 08:01:15,330 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:437210x0, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 08:01:15,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:43721-0x10075f501c50000 connected 2023-09-04 08:01:15,347 DEBUG [Listener at localhost/34091] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 08:01:15,347 DEBUG [Listener at localhost/34091] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 08:01:15,348 DEBUG [Listener at localhost/34091] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 08:01:15,348 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43721 2023-09-04 08:01:15,348 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43721 2023-09-04 08:01:15,348 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43721 2023-09-04 08:01:15,349 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43721 2023-09-04 08:01:15,349 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43721 2023-09-04 08:01:15,349 INFO [Listener at localhost/34091] master.HMaster(444): hbase.rootdir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8, hbase.cluster.distributed=false 2023-09-04 08:01:15,361 INFO [Listener at localhost/34091] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-09-04 08:01:15,361 INFO [Listener at localhost/34091] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 08:01:15,362 INFO [Listener at localhost/34091] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-09-04 08:01:15,362 INFO [Listener at localhost/34091] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-09-04 08:01:15,362 INFO [Listener at localhost/34091] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-09-04 08:01:15,362 INFO [Listener at localhost/34091] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-09-04 08:01:15,362 INFO [Listener at localhost/34091] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-09-04 08:01:15,364 INFO [Listener at localhost/34091] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42585 2023-09-04 08:01:15,364 INFO [Listener at localhost/34091] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-04 08:01:15,365 DEBUG [Listener at localhost/34091] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-09-04 08:01:15,366 INFO [Listener at localhost/34091] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 08:01:15,367 INFO [Listener at localhost/34091] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 08:01:15,367 INFO [Listener at localhost/34091] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42585 connecting to ZooKeeper ensemble=127.0.0.1:54397 2023-09-04 08:01:15,371 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): regionserver:425850x0, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-09-04 08:01:15,372 DEBUG [Listener at localhost/34091] zookeeper.ZKUtil(164): regionserver:425850x0, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 08:01:15,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42585-0x10075f501c50001 connected 2023-09-04 08:01:15,373 DEBUG [Listener at localhost/34091] zookeeper.ZKUtil(164): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 08:01:15,374 DEBUG [Listener at localhost/34091] zookeeper.ZKUtil(164): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-09-04 08:01:15,374 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42585 2023-09-04 08:01:15,374 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42585 2023-09-04 08:01:15,374 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42585 2023-09-04 08:01:15,375 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42585 2023-09-04 08:01:15,375 DEBUG [Listener at localhost/34091] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42585 2023-09-04 08:01:15,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:15,377 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 08:01:15,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:15,379 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 08:01:15,379 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-09-04 08:01:15,379 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:15,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 08:01:15,381 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43721,1693814475315 from backup master directory 2023-09-04 08:01:15,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-09-04 08:01:15,383 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:15,383 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 08:01:15,383 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:15,383 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-09-04 08:01:15,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/hbase.id with ID: 29264784-c839-46c5-ad85-b72d7ea7d335 2023-09-04 08:01:15,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 08:01:15,411 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:15,420 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4abc67a9 to 127.0.0.1:54397 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 08:01:15,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@59c90a50, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 08:01:15,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-09-04 08:01:15,425 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-09-04 08:01:15,425 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 08:01:15,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store-tmp 2023-09-04 08:01:15,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 08:01:15,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 08:01:15,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:15,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:15,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 08:01:15,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:15,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:15,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 08:01:15,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/WALs/jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:15,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43721%2C1693814475315, suffix=, logDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/WALs/jenkins-hbase4.apache.org,43721,1693814475315, archiveDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/oldWALs, maxLogs=10 2023-09-04 08:01:15,451 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK] 2023-09-04 08:01:15,451 DEBUG [RS-EventLoopGroup-10-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK] 2023-09-04 08:01:15,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/WALs/jenkins-hbase4.apache.org,43721,1693814475315/jenkins-hbase4.apache.org%2C43721%2C1693814475315.1693814475437 2023-09-04 08:01:15,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK], DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK]] 2023-09-04 08:01:15,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-09-04 08:01:15,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 08:01:15,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 08:01:15,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 08:01:15,456 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-09-04 08:01:15,457 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-09-04 08:01:15,457 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-09-04 08:01:15,458 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:01:15,458 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 08:01:15,459 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-09-04 08:01:15,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-09-04 08:01:15,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 08:01:15,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=884013, jitterRate=0.12408091127872467}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 08:01:15,464 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 08:01:15,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-09-04 08:01:15,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-09-04 08:01:15,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-09-04 08:01:15,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-09-04 08:01:15,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-09-04 08:01:15,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 0 msec 2023-09-04 08:01:15,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-09-04 08:01:15,467 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-09-04 08:01:15,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-09-04 08:01:15,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-09-04 08:01:15,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-09-04 08:01:15,478 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-09-04 08:01:15,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-09-04 08:01:15,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-09-04 08:01:15,481 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:15,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-09-04 08:01:15,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-09-04 08:01:15,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-09-04 08:01:15,487 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 08:01:15,487 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-09-04 08:01:15,487 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:15,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,43721,1693814475315, sessionid=0x10075f501c50000, setting cluster-up flag (Was=false) 2023-09-04 08:01:15,491 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:15,495 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-09-04 08:01:15,496 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:15,498 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:15,503 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-09-04 08:01:15,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:15,505 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/.hbase-snapshot/.tmp 2023-09-04 08:01:15,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-09-04 08:01:15,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 08:01:15,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 08:01:15,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 08:01:15,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-09-04 08:01:15,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-09-04 08:01:15,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 08:01:15,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,508 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1693814505508 2023-09-04 08:01:15,508 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-09-04 08:01:15,508 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-09-04 08:01:15,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-09-04 08:01:15,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-09-04 08:01:15,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-09-04 08:01:15,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-09-04 08:01:15,509 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 08:01:15,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,509 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-09-04 08:01:15,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-09-04 08:01:15,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-09-04 08:01:15,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-09-04 08:01:15,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-09-04 08:01:15,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-09-04 08:01:15,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814475510,5,FailOnTimeoutGroup] 2023-09-04 08:01:15,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814475510,5,FailOnTimeoutGroup] 2023-09-04 08:01:15,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-09-04 08:01:15,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,510 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 08:01:15,519 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 08:01:15,519 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-09-04 08:01:15,519 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8 2023-09-04 08:01:15,527 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 08:01:15,528 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 08:01:15,530 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/info 2023-09-04 08:01:15,530 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 08:01:15,531 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:01:15,531 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 08:01:15,532 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/rep_barrier 2023-09-04 08:01:15,533 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 08:01:15,533 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:01:15,533 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 08:01:15,534 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/table 2023-09-04 08:01:15,535 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 08:01:15,535 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:01:15,536 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740 2023-09-04 08:01:15,537 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740 2023-09-04 08:01:15,539 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 08:01:15,540 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 08:01:15,541 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 08:01:15,542 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=868133, jitterRate=0.10388891398906708}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 08:01:15,542 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 08:01:15,542 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 08:01:15,542 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 08:01:15,542 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 08:01:15,542 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 08:01:15,542 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 08:01:15,542 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 08:01:15,543 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 08:01:15,543 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-09-04 08:01:15,543 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-09-04 08:01:15,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-09-04 08:01:15,545 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-09-04 08:01:15,546 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-09-04 08:01:15,577 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(951): ClusterId : 29264784-c839-46c5-ad85-b72d7ea7d335 2023-09-04 08:01:15,578 DEBUG [RS:0;jenkins-hbase4:42585] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-09-04 08:01:15,580 DEBUG [RS:0;jenkins-hbase4:42585] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-09-04 08:01:15,580 DEBUG [RS:0;jenkins-hbase4:42585] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-09-04 08:01:15,583 DEBUG [RS:0;jenkins-hbase4:42585] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-09-04 08:01:15,583 DEBUG [RS:0;jenkins-hbase4:42585] zookeeper.ReadOnlyZKClient(139): Connect 0x3358d81e to 127.0.0.1:54397 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 08:01:15,587 DEBUG [RS:0;jenkins-hbase4:42585] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3d9f2005, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 08:01:15,587 DEBUG [RS:0;jenkins-hbase4:42585] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@39a0b36e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 08:01:15,595 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42585 2023-09-04 08:01:15,595 INFO [RS:0;jenkins-hbase4:42585] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-09-04 08:01:15,595 INFO [RS:0;jenkins-hbase4:42585] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-09-04 08:01:15,596 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1022): About to register with Master. 2023-09-04 08:01:15,596 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,43721,1693814475315 with isa=jenkins-hbase4.apache.org/172.31.14.131:42585, startcode=1693814475361 2023-09-04 08:01:15,596 DEBUG [RS:0;jenkins-hbase4:42585] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-09-04 08:01:15,598 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48161, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-09-04 08:01:15,599 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43721] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:15,599 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8 2023-09-04 08:01:15,600 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33613 2023-09-04 08:01:15,600 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-09-04 08:01:15,601 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 08:01:15,601 DEBUG [RS:0;jenkins-hbase4:42585] zookeeper.ZKUtil(162): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:15,601 WARN [RS:0;jenkins-hbase4:42585] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-09-04 08:01:15,602 INFO [RS:0;jenkins-hbase4:42585] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 08:01:15,602 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:15,602 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42585,1693814475361] 2023-09-04 08:01:15,606 DEBUG [RS:0;jenkins-hbase4:42585] zookeeper.ZKUtil(162): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:15,606 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-09-04 08:01:15,607 INFO [RS:0;jenkins-hbase4:42585] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-09-04 08:01:15,608 INFO [RS:0;jenkins-hbase4:42585] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-09-04 08:01:15,609 INFO [RS:0;jenkins-hbase4:42585] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-09-04 08:01:15,609 INFO [RS:0;jenkins-hbase4:42585] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,609 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-09-04 08:01:15,610 INFO [RS:0;jenkins-hbase4:42585] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,611 DEBUG [RS:0;jenkins-hbase4:42585] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-09-04 08:01:15,612 INFO [RS:0;jenkins-hbase4:42585] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,612 INFO [RS:0;jenkins-hbase4:42585] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,612 INFO [RS:0;jenkins-hbase4:42585] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,624 INFO [RS:0;jenkins-hbase4:42585] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-09-04 08:01:15,624 INFO [RS:0;jenkins-hbase4:42585] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42585,1693814475361-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,635 INFO [RS:0;jenkins-hbase4:42585] regionserver.Replication(203): jenkins-hbase4.apache.org,42585,1693814475361 started 2023-09-04 08:01:15,635 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,42585,1693814475361, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42585, sessionid=0x10075f501c50001 2023-09-04 08:01:15,635 DEBUG [RS:0;jenkins-hbase4:42585] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-09-04 08:01:15,635 DEBUG [RS:0;jenkins-hbase4:42585] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:15,635 DEBUG [RS:0;jenkins-hbase4:42585] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42585,1693814475361' 2023-09-04 08:01:15,635 DEBUG [RS:0;jenkins-hbase4:42585] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-09-04 08:01:15,636 DEBUG [RS:0;jenkins-hbase4:42585] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-09-04 08:01:15,636 DEBUG [RS:0;jenkins-hbase4:42585] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-09-04 08:01:15,636 DEBUG [RS:0;jenkins-hbase4:42585] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-09-04 08:01:15,636 DEBUG [RS:0;jenkins-hbase4:42585] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:15,636 DEBUG [RS:0;jenkins-hbase4:42585] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42585,1693814475361' 2023-09-04 08:01:15,636 DEBUG [RS:0;jenkins-hbase4:42585] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-09-04 08:01:15,636 DEBUG [RS:0;jenkins-hbase4:42585] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-09-04 08:01:15,636 DEBUG [RS:0;jenkins-hbase4:42585] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-09-04 08:01:15,636 INFO [RS:0;jenkins-hbase4:42585] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-09-04 08:01:15,636 INFO [RS:0;jenkins-hbase4:42585] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-09-04 08:01:15,696 DEBUG [jenkins-hbase4:43721] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-09-04 08:01:15,697 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42585,1693814475361, state=OPENING 2023-09-04 08:01:15,698 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-09-04 08:01:15,700 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:15,701 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 08:01:15,701 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42585,1693814475361}] 2023-09-04 08:01:15,738 INFO [RS:0;jenkins-hbase4:42585] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42585%2C1693814475361, suffix=, logDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/jenkins-hbase4.apache.org,42585,1693814475361, archiveDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/oldWALs, maxLogs=32 2023-09-04 08:01:15,753 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK] 2023-09-04 08:01:15,753 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK] 2023-09-04 08:01:15,755 INFO [RS:0;jenkins-hbase4:42585] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/jenkins-hbase4.apache.org,42585,1693814475361/jenkins-hbase4.apache.org%2C42585%2C1693814475361.1693814475739 2023-09-04 08:01:15,755 DEBUG [RS:0;jenkins-hbase4:42585] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK], DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK]] 2023-09-04 08:01:15,854 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:15,855 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-09-04 08:01:15,857 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38304, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-09-04 08:01:15,860 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-09-04 08:01:15,860 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 08:01:15,862 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42585%2C1693814475361.meta, suffix=.meta, logDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/jenkins-hbase4.apache.org,42585,1693814475361, archiveDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/oldWALs, maxLogs=32 2023-09-04 08:01:15,876 DEBUG [RS-EventLoopGroup-10-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK] 2023-09-04 08:01:15,877 DEBUG [RS-EventLoopGroup-10-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK] 2023-09-04 08:01:15,879 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/jenkins-hbase4.apache.org,42585,1693814475361/jenkins-hbase4.apache.org%2C42585%2C1693814475361.meta.1693814475862.meta 2023-09-04 08:01:15,879 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK], DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK]] 2023-09-04 08:01:15,879 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-09-04 08:01:15,880 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-09-04 08:01:15,880 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-09-04 08:01:15,880 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-09-04 08:01:15,880 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-09-04 08:01:15,880 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 08:01:15,880 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-09-04 08:01:15,880 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-09-04 08:01:15,881 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-09-04 08:01:15,882 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/info 2023-09-04 08:01:15,882 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/info 2023-09-04 08:01:15,882 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-09-04 08:01:15,883 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:01:15,883 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-09-04 08:01:15,883 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/rep_barrier 2023-09-04 08:01:15,884 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/rep_barrier 2023-09-04 08:01:15,884 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-09-04 08:01:15,884 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:01:15,884 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-09-04 08:01:15,885 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/table 2023-09-04 08:01:15,885 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/table 2023-09-04 08:01:15,886 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-09-04 08:01:15,886 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:01:15,887 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740 2023-09-04 08:01:15,888 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740 2023-09-04 08:01:15,890 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (16.0 M)) instead. 2023-09-04 08:01:15,891 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-09-04 08:01:15,892 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=842774, jitterRate=0.07164302468299866}}}, FlushLargeStoresPolicy{flushSizeLowerBound=16777216} 2023-09-04 08:01:15,892 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-09-04 08:01:15,892 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1693814475854 2023-09-04 08:01:15,896 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-09-04 08:01:15,896 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-09-04 08:01:15,897 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42585,1693814475361, state=OPEN 2023-09-04 08:01:15,898 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-09-04 08:01:15,898 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-09-04 08:01:15,900 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-09-04 08:01:15,900 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42585,1693814475361 in 197 msec 2023-09-04 08:01:15,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-09-04 08:01:15,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 357 msec 2023-09-04 08:01:15,902 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 397 msec 2023-09-04 08:01:15,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1693814475902, completionTime=-1 2023-09-04 08:01:15,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-09-04 08:01:15,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-09-04 08:01:15,905 DEBUG [hconnection-0x6e4f4d27-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 08:01:15,906 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38314, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 08:01:15,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-09-04 08:01:15,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1693814535908 2023-09-04 08:01:15,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1693814595908 2023-09-04 08:01:15,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-09-04 08:01:15,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43721,1693814475315-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43721,1693814475315-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43721,1693814475315-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43721, period=300000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-09-04 08:01:15,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-09-04 08:01:15,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-09-04 08:01:15,915 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-09-04 08:01:15,915 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-09-04 08:01:15,916 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-09-04 08:01:15,917 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-09-04 08:01:15,918 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/.tmp/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:15,919 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/.tmp/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e empty. 2023-09-04 08:01:15,919 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/.tmp/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:15,919 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-09-04 08:01:15,929 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-09-04 08:01:15,930 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 36553531886b5d6c83f39fd6b7c3271e, NAME => 'hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/.tmp 2023-09-04 08:01:15,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 08:01:15,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 36553531886b5d6c83f39fd6b7c3271e, disabling compactions & flushes 2023-09-04 08:01:15,938 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:15,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:15,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. after waiting 0 ms 2023-09-04 08:01:15,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:15,938 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:15,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 36553531886b5d6c83f39fd6b7c3271e: 2023-09-04 08:01:15,940 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-09-04 08:01:15,941 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814475941"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1693814475941"}]},"ts":"1693814475941"} 2023-09-04 08:01:15,943 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-09-04 08:01:15,944 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-09-04 08:01:15,944 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814475944"}]},"ts":"1693814475944"} 2023-09-04 08:01:15,945 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-09-04 08:01:15,954 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=36553531886b5d6c83f39fd6b7c3271e, ASSIGN}] 2023-09-04 08:01:15,956 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=36553531886b5d6c83f39fd6b7c3271e, ASSIGN 2023-09-04 08:01:15,956 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=36553531886b5d6c83f39fd6b7c3271e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42585,1693814475361; forceNewPlan=false, retain=false 2023-09-04 08:01:16,107 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=36553531886b5d6c83f39fd6b7c3271e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:16,107 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814476107"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1693814476107"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1693814476107"}]},"ts":"1693814476107"} 2023-09-04 08:01:16,109 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 36553531886b5d6c83f39fd6b7c3271e, server=jenkins-hbase4.apache.org,42585,1693814475361}] 2023-09-04 08:01:16,264 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:16,265 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36553531886b5d6c83f39fd6b7c3271e, NAME => 'hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e.', STARTKEY => '', ENDKEY => ''} 2023-09-04 08:01:16,265 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,265 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-09-04 08:01:16,265 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,265 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,266 INFO [StoreOpener-36553531886b5d6c83f39fd6b7c3271e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,267 DEBUG [StoreOpener-36553531886b5d6c83f39fd6b7c3271e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e/info 2023-09-04 08:01:16,268 DEBUG [StoreOpener-36553531886b5d6c83f39fd6b7c3271e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e/info 2023-09-04 08:01:16,268 INFO [StoreOpener-36553531886b5d6c83f39fd6b7c3271e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36553531886b5d6c83f39fd6b7c3271e columnFamilyName info 2023-09-04 08:01:16,268 INFO [StoreOpener-36553531886b5d6c83f39fd6b7c3271e-1] regionserver.HStore(310): Store=36553531886b5d6c83f39fd6b7c3271e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-09-04 08:01:16,269 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,270 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,272 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,275 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-09-04 08:01:16,275 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 36553531886b5d6c83f39fd6b7c3271e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=16384, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=884182, jitterRate=0.12429645657539368}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-09-04 08:01:16,275 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 36553531886b5d6c83f39fd6b7c3271e: 2023-09-04 08:01:16,276 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e., pid=6, masterSystemTime=1693814476261 2023-09-04 08:01:16,277 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:16,278 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:16,278 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=36553531886b5d6c83f39fd6b7c3271e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:16,278 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1693814476278"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1693814476278"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1693814476278"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1693814476278"}]},"ts":"1693814476278"} 2023-09-04 08:01:16,281 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-09-04 08:01:16,281 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 36553531886b5d6c83f39fd6b7c3271e, server=jenkins-hbase4.apache.org,42585,1693814475361 in 170 msec 2023-09-04 08:01:16,283 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-09-04 08:01:16,283 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=36553531886b5d6c83f39fd6b7c3271e, ASSIGN in 327 msec 2023-09-04 08:01:16,284 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-09-04 08:01:16,284 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1693814476284"}]},"ts":"1693814476284"} 2023-09-04 08:01:16,285 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-09-04 08:01:16,287 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-09-04 08:01:16,289 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 373 msec 2023-09-04 08:01:16,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-09-04 08:01:16,317 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-09-04 08:01:16,317 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:16,321 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-09-04 08:01:16,329 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 08:01:16,332 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 10 msec 2023-09-04 08:01:16,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-09-04 08:01:16,351 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-09-04 08:01:16,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 10 msec 2023-09-04 08:01:16,371 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-09-04 08:01:16,373 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-09-04 08:01:16,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 0.990sec 2023-09-04 08:01:16,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-09-04 08:01:16,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-09-04 08:01:16,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-09-04 08:01:16,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43721,1693814475315-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-09-04 08:01:16,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43721,1693814475315-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-09-04 08:01:16,376 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-09-04 08:01:16,377 DEBUG [Listener at localhost/34091] zookeeper.ReadOnlyZKClient(139): Connect 0x04df8908 to 127.0.0.1:54397 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-09-04 08:01:16,385 DEBUG [Listener at localhost/34091] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77d6f2f1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-09-04 08:01:16,387 DEBUG [hconnection-0x5c680157-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-09-04 08:01:16,389 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38322, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-09-04 08:01:16,391 INFO [Listener at localhost/34091] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:16,392 INFO [Listener at localhost/34091] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-09-04 08:01:16,395 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-09-04 08:01:16,395 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:16,396 INFO [Listener at localhost/34091] master.MasterRpcServices(492): Client=null/null set balanceSwitch=false 2023-09-04 08:01:16,396 INFO [Listener at localhost/34091] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-09-04 08:01:16,399 INFO [Listener at localhost/34091] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=test.com%2C8080%2C1, suffix=, logDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/test.com,8080,1, archiveDir=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/oldWALs, maxLogs=32 2023-09-04 08:01:16,420 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK] 2023-09-04 08:01:16,421 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK] 2023-09-04 08:01:16,423 INFO [Listener at localhost/34091] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/test.com,8080,1/test.com%2C8080%2C1.1693814476399 2023-09-04 08:01:16,425 DEBUG [Listener at localhost/34091] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK], DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK]] 2023-09-04 08:01:16,439 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK] 2023-09-04 08:01:16,440 DEBUG [AsyncFSWAL-11-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = /127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK] 2023-09-04 08:01:16,442 INFO [Listener at localhost/34091] wal.AbstractFSWAL(802): Rolled WAL /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/test.com,8080,1/test.com%2C8080%2C1.1693814476399 with entries=0, filesize=83 B; new WAL /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/test.com,8080,1/test.com%2C8080%2C1.1693814476425 2023-09-04 08:01:16,442 DEBUG [Listener at localhost/34091] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:32899,DS-a557ff79-0a68-42cb-ba1f-ff2237e5c621,DISK], DatanodeInfoWithStorage[127.0.0.1:43333,DS-ad082d6e-0c82-4b47-ba3c-c1a047e737a2,DISK]] 2023-09-04 08:01:16,442 DEBUG [Listener at localhost/34091] wal.AbstractFSWAL(716): hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/test.com,8080,1/test.com%2C8080%2C1.1693814476399 is not closed yet, will try archiving it next time 2023-09-04 08:01:16,448 INFO [WAL-Archive-0] wal.AbstractFSWAL(783): Archiving hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/WALs/test.com,8080,1/test.com%2C8080%2C1.1693814476399 to hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/oldWALs/test.com%2C8080%2C1.1693814476399 2023-09-04 08:01:16,451 DEBUG [Listener at localhost/34091] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/oldWALs 2023-09-04 08:01:16,451 INFO [Listener at localhost/34091] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL test.com%2C8080%2C1:(num 1693814476425) 2023-09-04 08:01:16,451 INFO [Listener at localhost/34091] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-09-04 08:01:16,451 DEBUG [Listener at localhost/34091] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x04df8908 to 127.0.0.1:54397 2023-09-04 08:01:16,451 DEBUG [Listener at localhost/34091] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:16,452 DEBUG [Listener at localhost/34091] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-09-04 08:01:16,452 DEBUG [Listener at localhost/34091] util.JVMClusterUtil(257): Found active master hash=871982458, stopped=false 2023-09-04 08:01:16,452 INFO [Listener at localhost/34091] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:16,455 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 08:01:16,455 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-09-04 08:01:16,455 INFO [Listener at localhost/34091] procedure2.ProcedureExecutor(628): Stopping 2023-09-04 08:01:16,455 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:16,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 08:01:16,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-09-04 08:01:16,461 DEBUG [Listener at localhost/34091] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4abc67a9 to 127.0.0.1:54397 2023-09-04 08:01:16,461 DEBUG [Listener at localhost/34091] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:16,461 INFO [Listener at localhost/34091] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,42585,1693814475361' ***** 2023-09-04 08:01:16,461 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1064): Closing user regions 2023-09-04 08:01:16,461 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(3305): Received CLOSE for 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,461 INFO [Listener at localhost/34091] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-09-04 08:01:16,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 36553531886b5d6c83f39fd6b7c3271e, disabling compactions & flushes 2023-09-04 08:01:16,462 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:16,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:16,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. after waiting 0 ms 2023-09-04 08:01:16,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:16,462 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 36553531886b5d6c83f39fd6b7c3271e 1/1 column families, dataSize=78 B heapSize=488 B 2023-09-04 08:01:16,463 INFO [RS:0;jenkins-hbase4:42585] regionserver.HeapMemoryManager(220): Stopping 2023-09-04 08:01:16,463 INFO [RS:0;jenkins-hbase4:42585] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-09-04 08:01:16,463 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-09-04 08:01:16,463 INFO [RS:0;jenkins-hbase4:42585] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-09-04 08:01:16,464 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:16,464 DEBUG [RS:0;jenkins-hbase4:42585] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3358d81e to 127.0.0.1:54397 2023-09-04 08:01:16,464 DEBUG [RS:0;jenkins-hbase4:42585] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:16,464 INFO [RS:0;jenkins-hbase4:42585] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-09-04 08:01:16,464 INFO [RS:0;jenkins-hbase4:42585] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-09-04 08:01:16,464 INFO [RS:0;jenkins-hbase4:42585] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-09-04 08:01:16,464 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-09-04 08:01:16,464 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-09-04 08:01:16,464 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1478): Online Regions={36553531886b5d6c83f39fd6b7c3271e=hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e., 1588230740=hbase:meta,,1.1588230740} 2023-09-04 08:01:16,465 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1504): Waiting on 1588230740, 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,466 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-09-04 08:01:16,467 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-09-04 08:01:16,468 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-09-04 08:01:16,468 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-09-04 08:01:16,469 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-09-04 08:01:16,469 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=1.26 KB heapSize=2.89 KB 2023-09-04 08:01:16,491 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.17 KB at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/.tmp/info/c435ed8374a04f898c2d0bff7ab0c152 2023-09-04 08:01:16,510 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=94 B at sequenceid=9 (bloomFilter=false), to=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/.tmp/table/fbd8c4f9a06548b3bdc0bec58d850389 2023-09-04 08:01:16,516 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/.tmp/info/c435ed8374a04f898c2d0bff7ab0c152 as hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/info/c435ed8374a04f898c2d0bff7ab0c152 2023-09-04 08:01:16,522 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/info/c435ed8374a04f898c2d0bff7ab0c152, entries=10, sequenceid=9, filesize=5.9 K 2023-09-04 08:01:16,523 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/.tmp/table/fbd8c4f9a06548b3bdc0bec58d850389 as hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/table/fbd8c4f9a06548b3bdc0bec58d850389 2023-09-04 08:01:16,528 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/table/fbd8c4f9a06548b3bdc0bec58d850389, entries=2, sequenceid=9, filesize=4.7 K 2023-09-04 08:01:16,530 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~1.26 KB/1290, heapSize ~2.61 KB/2672, currentSize=0 B/0 for 1588230740 in 61ms, sequenceid=9, compaction requested=false 2023-09-04 08:01:16,542 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/meta/1588230740/recovered.edits/12.seqid, newMaxSeqId=12, maxSeqId=1 2023-09-04 08:01:16,543 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-09-04 08:01:16,543 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-09-04 08:01:16,543 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-09-04 08:01:16,543 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-09-04 08:01:16,667 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1504): Waiting on 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,687 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-09-04 08:01:16,687 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-09-04 08:01:16,868 DEBUG [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1504): Waiting on 36553531886b5d6c83f39fd6b7c3271e 2023-09-04 08:01:16,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e/.tmp/info/31efbc635a2947d1b344b3e58c3409f9 2023-09-04 08:01:16,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e/.tmp/info/31efbc635a2947d1b344b3e58c3409f9 as hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e/info/31efbc635a2947d1b344b3e58c3409f9 2023-09-04 08:01:16,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e/info/31efbc635a2947d1b344b3e58c3409f9, entries=2, sequenceid=6, filesize=4.8 K 2023-09-04 08:01:16,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 36553531886b5d6c83f39fd6b7c3271e in 440ms, sequenceid=6, compaction requested=false 2023-09-04 08:01:16,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/data/hbase/namespace/36553531886b5d6c83f39fd6b7c3271e/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-09-04 08:01:16,907 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:16,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 36553531886b5d6c83f39fd6b7c3271e: 2023-09-04 08:01:16,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1693814475914.36553531886b5d6c83f39fd6b7c3271e. 2023-09-04 08:01:17,068 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,42585,1693814475361; all regions closed. 2023-09-04 08:01:17,073 DEBUG [RS:0;jenkins-hbase4:42585] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/oldWALs 2023-09-04 08:01:17,073 INFO [RS:0;jenkins-hbase4:42585] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42585%2C1693814475361.meta:.meta(num 1693814475862) 2023-09-04 08:01:17,078 DEBUG [RS:0;jenkins-hbase4:42585] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/oldWALs 2023-09-04 08:01:17,078 INFO [RS:0;jenkins-hbase4:42585] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42585%2C1693814475361:(num 1693814475739) 2023-09-04 08:01:17,078 DEBUG [RS:0;jenkins-hbase4:42585] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:17,078 INFO [RS:0;jenkins-hbase4:42585] regionserver.LeaseManager(133): Closed leases 2023-09-04 08:01:17,078 INFO [RS:0;jenkins-hbase4:42585] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-09-04 08:01:17,078 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 08:01:17,079 INFO [RS:0;jenkins-hbase4:42585] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42585 2023-09-04 08:01:17,081 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42585,1693814475361 2023-09-04 08:01:17,081 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 08:01:17,081 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-09-04 08:01:17,083 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42585,1693814475361] 2023-09-04 08:01:17,083 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42585,1693814475361; numProcessing=1 2023-09-04 08:01:17,084 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42585,1693814475361 already deleted, retry=false 2023-09-04 08:01:17,084 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42585,1693814475361 expired; onlineServers=0 2023-09-04 08:01:17,084 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43721,1693814475315' ***** 2023-09-04 08:01:17,084 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-09-04 08:01:17,085 DEBUG [M:0;jenkins-hbase4:43721] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37c6f7dd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-09-04 08:01:17,085 INFO [M:0;jenkins-hbase4:43721] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43721,1693814475315 2023-09-04 08:01:17,085 INFO [M:0;jenkins-hbase4:43721] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43721,1693814475315; all regions closed. 2023-09-04 08:01:17,085 DEBUG [M:0;jenkins-hbase4:43721] ipc.AbstractRpcClient(494): Stopping rpc client 2023-09-04 08:01:17,085 DEBUG [M:0;jenkins-hbase4:43721] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-09-04 08:01:17,085 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-09-04 08:01:17,085 DEBUG [M:0;jenkins-hbase4:43721] cleaner.HFileCleaner(317): Stopping file delete threads 2023-09-04 08:01:17,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814475510] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1693814475510,5,FailOnTimeoutGroup] 2023-09-04 08:01:17,085 INFO [M:0;jenkins-hbase4:43721] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-09-04 08:01:17,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814475510] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1693814475510,5,FailOnTimeoutGroup] 2023-09-04 08:01:17,086 INFO [M:0;jenkins-hbase4:43721] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-09-04 08:01:17,086 INFO [M:0;jenkins-hbase4:43721] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-09-04 08:01:17,086 DEBUG [M:0;jenkins-hbase4:43721] master.HMaster(1512): Stopping service threads 2023-09-04 08:01:17,086 INFO [M:0;jenkins-hbase4:43721] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-09-04 08:01:17,087 ERROR [M:0;jenkins-hbase4:43721] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] 2023-09-04 08:01:17,087 INFO [M:0;jenkins-hbase4:43721] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-09-04 08:01:17,087 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-09-04 08:01:17,088 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-09-04 08:01:17,088 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-09-04 08:01:17,088 DEBUG [M:0;jenkins-hbase4:43721] zookeeper.ZKUtil(398): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-09-04 08:01:17,088 WARN [M:0;jenkins-hbase4:43721] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-09-04 08:01:17,088 INFO [M:0;jenkins-hbase4:43721] assignment.AssignmentManager(315): Stopping assignment manager 2023-09-04 08:01:17,088 INFO [M:0;jenkins-hbase4:43721] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-09-04 08:01:17,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-09-04 08:01:17,088 DEBUG [M:0;jenkins-hbase4:43721] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-09-04 08:01:17,088 INFO [M:0;jenkins-hbase4:43721] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:17,088 DEBUG [M:0;jenkins-hbase4:43721] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:17,088 DEBUG [M:0;jenkins-hbase4:43721] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-09-04 08:01:17,089 DEBUG [M:0;jenkins-hbase4:43721] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:17,089 INFO [M:0;jenkins-hbase4:43721] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=24.07 KB heapSize=29.55 KB 2023-09-04 08:01:17,100 INFO [M:0;jenkins-hbase4:43721] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=24.07 KB at sequenceid=66 (bloomFilter=true), to=hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/77d86d66f31e42dabf20dc4d5b4d1caa 2023-09-04 08:01:17,106 DEBUG [M:0;jenkins-hbase4:43721] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/77d86d66f31e42dabf20dc4d5b4d1caa as hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/77d86d66f31e42dabf20dc4d5b4d1caa 2023-09-04 08:01:17,111 INFO [M:0;jenkins-hbase4:43721] regionserver.HStore(1080): Added hdfs://localhost:33613/user/jenkins/test-data/0cb52bd5-7a3f-7d14-6155-b578f7908ef8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/77d86d66f31e42dabf20dc4d5b4d1caa, entries=8, sequenceid=66, filesize=6.3 K 2023-09-04 08:01:17,112 INFO [M:0;jenkins-hbase4:43721] regionserver.HRegion(2948): Finished flush of dataSize ~24.07 KB/24646, heapSize ~29.54 KB/30248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=66, compaction requested=false 2023-09-04 08:01:17,114 INFO [M:0;jenkins-hbase4:43721] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-09-04 08:01:17,114 DEBUG [M:0;jenkins-hbase4:43721] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-09-04 08:01:17,119 INFO [M:0;jenkins-hbase4:43721] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-09-04 08:01:17,119 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-09-04 08:01:17,119 INFO [M:0;jenkins-hbase4:43721] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43721 2023-09-04 08:01:17,122 DEBUG [M:0;jenkins-hbase4:43721] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43721,1693814475315 already deleted, retry=false 2023-09-04 08:01:17,183 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 08:01:17,183 INFO [RS:0;jenkins-hbase4:42585] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,42585,1693814475361; zookeeper connection closed. 2023-09-04 08:01:17,183 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): regionserver:42585-0x10075f501c50001, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 08:01:17,184 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2058180] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2058180 2023-09-04 08:01:17,184 INFO [Listener at localhost/34091] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-09-04 08:01:17,283 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 08:01:17,283 INFO [M:0;jenkins-hbase4:43721] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43721,1693814475315; zookeeper connection closed. 2023-09-04 08:01:17,283 DEBUG [Listener at localhost/34091-EventThread] zookeeper.ZKWatcher(600): master:43721-0x10075f501c50000, quorum=127.0.0.1:54397, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-09-04 08:01:17,285 WARN [Listener at localhost/34091] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 08:01:17,288 INFO [Listener at localhost/34091] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 08:01:17,392 WARN [BP-24500603-172.31.14.131-1693814474757 heartbeating to localhost/127.0.0.1:33613] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 08:01:17,392 WARN [BP-24500603-172.31.14.131-1693814474757 heartbeating to localhost/127.0.0.1:33613] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-24500603-172.31.14.131-1693814474757 (Datanode Uuid e42dc467-e8c1-4a4a-a472-fe6bd944c9cc) service to localhost/127.0.0.1:33613 2023-09-04 08:01:17,392 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/cluster_7d658e7b-cc6f-6476-4fa5-eba95b181060/dfs/data/data3/current/BP-24500603-172.31.14.131-1693814474757] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 08:01:17,393 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/cluster_7d658e7b-cc6f-6476-4fa5-eba95b181060/dfs/data/data4/current/BP-24500603-172.31.14.131-1693814474757] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 08:01:17,393 WARN [Listener at localhost/34091] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-09-04 08:01:17,396 INFO [Listener at localhost/34091] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 08:01:17,500 WARN [BP-24500603-172.31.14.131-1693814474757 heartbeating to localhost/127.0.0.1:33613] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-09-04 08:01:17,500 WARN [BP-24500603-172.31.14.131-1693814474757 heartbeating to localhost/127.0.0.1:33613] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-24500603-172.31.14.131-1693814474757 (Datanode Uuid d42ac892-09b9-4145-b42e-55e763d7a8d2) service to localhost/127.0.0.1:33613 2023-09-04 08:01:17,501 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/cluster_7d658e7b-cc6f-6476-4fa5-eba95b181060/dfs/data/data1/current/BP-24500603-172.31.14.131-1693814474757] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 08:01:17,501 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4c7a1d68-9f9f-2435-aa25-4c3161e999e8/cluster_7d658e7b-cc6f-6476-4fa5-eba95b181060/dfs/data/data2/current/BP-24500603-172.31.14.131-1693814474757] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-09-04 08:01:17,512 INFO [Listener at localhost/34091] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-09-04 08:01:17,613 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-09-04 08:01:17,623 INFO [Listener at localhost/34091] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-09-04 08:01:17,635 INFO [Listener at localhost/34091] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-09-04 08:01:17,645 INFO [Listener at localhost/34091] hbase.ResourceChecker(175): after: regionserver.wal.TestAsyncLogRolling#testLogRollOnNothingWritten Thread=99 (was 76) - Thread LEAK? -, OpenFileDescriptor=494 (was 464) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=101 (was 56) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=8185 (was 8197)